First it is good to note that categories will always appear before the products on the site page.
The categories can be sorted/ordered by going to Sites>Categories>General Information tab. You will see a "Sort Order" field that allows you to order your categories.
The products can also be sorted. Go to Sites >Products>General Tab.
Both will first be sorted by the number given, and then appear in alphabetical order.