I think the problem is mostly that I saw the movie first - I bet I would have been a lot more impressed with the book if I hadn't. But the book is quite short and sparse (which is understandable, given the circumstances of its creation), so it comes across as rather barren, in contrast to the lush visuals of the film. I'm impressed that the makers of the movie could conjure up such incredible imagery on the basis of the text. This could also be a translation issue - the movie is in Bauby's native French, but I read the book in English. Certainly, there are parts of the book that are explicitly quoted in the text, but they seem to be the highlights, honestly.
There's also the difference between the first person of the text and the sort of indirect discourse of the cinematic perspective, which allows you a bit of distance from the protagonist. This distance, I think, gives you a bit of extra space to appreciate the narrative construction. Though it must also be said that the book lacks a real narrative construction - rather than an overarching plot arch, it reads like a collection of fragments. Again, the mode of construction more than justifies this, and I'm not faulting the guy, or diminishing his achievement, but at the end of the day, the film is able to do more with the material.
There is one thing in the book that I really loved, and I don't remember if it's in the film. It's a description of a trip Bauby took with a lover, and things weren't really going so well with the two of them, and to make matters worse, he was completely immersed in a book and was having a hard time tearing himself away from it while they were traveling. Didn't want to get out of the car and see the sights, didn't want to do anything but read. I loved it, because it's an experience I've had many times myself. And it's a horribly selfish thing to do, and absolutely maddening to the people around you. And furthermore, it's ridiculous, because you're missing out on these great views and sights and things that you might not get a chance to see again in favor of a book you can just as well read tomorrow or the next day, but at that particular moment, when you're sucked into a good book, it seems absolutely worth it. And you know, even looking back on it later, you don't regret having chosen the book, even though you know you probably should. In this case, it's exacerbated by the fact that he's not only missing out on the joys of the trip, he's also neglecting this woman, who takes her revenge, fittingly, on the book. But still, he gets the final word, because thankfully, she's only desecrated pages that he's already read.