I have published an update (3.14.1) which allows you to permanently hide the in-page button.

To disable the button in the page, right-click on the extension button in the address bar and toggle the "Show Lyrics Here button in the page." menu item (this instruction also appears in the tooltip of the in-page Lyrics Here button).