A.2.7 Point is wrong in the COMMIT_EDITMSG buffer ¶

Neither Magit nor git-commit.el fiddle with point in the buffer used to write commit messages, so something else must be doing it.

You have probably globally enabled a mode which restores point in file-visiting buffers. It might be a bit surprising, but when you write a commit message, then you are actually editing a file.

So you have to figure out which package is doing it. saveplace, pointback, and session are likely candidates. These snippets might help:

(setq session-name-disable-regexp "\\(?:\\`'\\.git/[A-Z_]+\\'\\)")

(with-eval-after-load 'pointback
  (lambda ()
    (when (or git-commit-mode git-rebase-mode)
      (pointback-mode -1))))