I found that the orderby parameter of gallery in 2.5 was broken. An incorrect SQL statement was being produced that meant that any orderby was ignored. This is now fixed in 2.5.1 and introduces a new value: ‘orderby=RAND()’ for random ordering.
See this trac entry for more.