Fish detect sound primarily through particle motion, which creates a 180° ambiguity in determining sound direction. Some fishes with gas-filled structures closely associated with the inner ear such as the swim bladder can also sense acoustic pressure. Among these pressure-sensitive species, some have been shown to resolve this ambiguity by comparing the phase relationship between pressure and particle motion, which is opposite for sound arriving from opposite directions. However, the biomechanical basis by which these phase differences produce distinct motions of the otolith end organs remains unclear. To address this gap, finite element modeling was used to simulate motion of the right saccular otolith (RSO) in female plainfin midshipman (Porichthys notatus). Without a swim bladder, the RSO exhibited identical back-and-forth displacements at 100 Hz for sounds arriving from either side, preserving directional ambiguity. When the swim bladder was included, the RSO followed elliptical orbits whose handedness depended on sound direction. Reversing the phase relationship between acoustic pressure and particle motion also reversed handedness of RSO motion, effectively making the sound source appear to originate from the opposite direction. These results provide a mechanistic explanation for how swim bladders may enable fishes to resolve the 180° ambiguity in directional hearing.