Personally, I think that if one can't connect with In a Priest Driven Ambulance (With Silver Sunshine Stares), then it's a merry chase to go hunting down everything else. That said, there are a a handful of anomalies which rarely get mentioned. The Southern Oklahoma Cosmic Trigger Contest is a soundtrack to a documentary titled Okie Noodling. It's fairly short and unlike anything in their library. Not the easiest to find, but if you enjoy rustic, country-ish songs supposedly about noodling, but instead get ribald tunes about randy adolescents, then you can't go wrong. Astounding guitaring and pianoing via Steven & Ronald.
The Soft Bulletin Companion collects alternate mixes from TSB, and mixed-down versions of tracks from Zaireeka. In my estimation it has one of the Lips saddest songs, "35,000 ft of Despair," and one of their most gloriously symphonic and bombastic ones "Satellite of You."
The Yeah, I Know It's a Drag...But Wastin' Pigs is Still Radicalep is dope. Though it should be retitled to Yeah, I Know It's a Drag...But Hanging out With Lana Del Rey and Her Cop Boyfriend Isn't Radical . There's an Echo & The Bunnymen cover, and this cool unreleased song
Sludgy artifact from the Sub Pop Singles Club finds them reworking "Drug Machine in Heaven" and the b-side covering "Strychnine" and "(What's So Funny 'Bout) Peace, Love, and Understanding."
And covering The Moles just because
But if you don't like In a Priest Driven Ambulance there's a good chance you won't like any of it, so I'd start there.