"putting it another way - why does the Preceeding column looks correct (from 1 to 10) while the the Following column ~has another ordering?"
Whenever you do a sort, if there are ties, the ties can come out in any order. You have four sorts in that query (the three OVER clauses and then the final ORDER BY). As long as there are ties, you have no control over the order of the tied items. They could all be the same in each sort, or they could be different. They could be the same the first time you run the query and then be different the next time you run the query.
If you care about the the order of tied items, you must provide enough info in the ORDER BY to be sure there are no ties. The usual way to do that is to use the Primary Keys. So, for example, if the primary key of your Inventory table is Warehouse, Product, BinNbr, then you would want to have those in your ORDER BY's. You already have Product, so just add the other keys and your query would look like
Select warehouse, Product, quantity,
Preceding = Count(*) Over(Order By Product, Warehouse, BinNbr Rows Between Unbounded Preceding And Current Row),
[Three current] = Count(*) Over(Order By Product, Warehouse, BinNbr Rows Between 1 Preceding And 1 Following),
Preceding = Count(*) Over(Order By Product, Warehouse, BinNbr Rows Between Current Row And Unbounded Following)
From Inventory
Order By Product, Warehouse, BinNbr;
Tom