When you have different page layouts (margins, background, page number, header and footer), you must use different page styles. A page style records the “geometry” of the page and also various properties like alphabetical/numeric page numbering, header contents, mirroring, …
A page style is active between two “boundaries”. This boundary is created by a special form of page break with Insert> More Breaks>Manual Break where you can choose the page style to activate after the break and the starting page number if needed.
Therefore, to solve your problem, use built-in First Page page style for your cover. This style automatically switches to Default Page Style at end of page. Consequently, you’ll probably have to create a user page style for your front matter (TOC and others) as Default Page Style is usually fit for bulk contents of the document (but you are free to follow another convention).
Insert a special break wherever you need to change page styles.
The special page break allows to define a starting page number. There is then absolutely no need to tweak the field when you Insert>Field>Page Number in the header or footer.
Offset is a special function to change the referenced page. By default the fields point to the current page (the cross-references point to the target page). By specifying an Offset, you shift the reference to a preceding (negative value) or following (positive value) page. When you’re near the start or end of your document, this page may not exist. In which case, the field returns “void”. This function is meant as an extension to text like “see previous/next page” where you can designate more remote pages. So, don’t use Offset unless you know what you’re doing.
Cross-references always use the “intrinsic” page number of the target. By “intrinsic”, I mean the page number as defined by the page style. This illustrates the fact that Offset is not a tool to modify the “intrinsic” page number.
Remark: in your question you use page count. I think you meant page number in fact. Page count is the total number of pages in your document. Presently there is no way to exclude parts of document from page counting. It can be worked around with a tricky use of cross-references.
A final note: on this site always mention in the question OS name, LO version and save format as there are subtle differences between platforms and releases. Also, stable and reliable formatting is guaranteed only in native format.
