When you configured Outlook you may have seen a msg on the lines of 'set outlook as default mail handler' but there would not have been any msg to set it as a start up.
To set Outlook as a start up you would have needed to manually add the outlook shortcut to the Start Up folder.
Its posible that a third party outlook add-in has created a win autostart, but I've never seen any outlook add-in do this.
Your statement that outlook is not run at log-in or pwer on of win is at odds with you stating that Outlook is running at log-on.
So when you power on is outlook actually running? or do you perhaps mean that you start outlook, then later close it, but its still a running process?