I think you explained it very well. I am familiar with the feeling (although I must confess to a good chunk of physical shallowness); I think that the common term for it is "fluid sexuality." It can be very confusing, but it can also be fun as you never really know what (or who) you're going to find yourself attracted to next.
