I’d like to start a collection of famous answers to this eminently ignorant question.
The first two I’ll start with are from Umberto Eco, as related by Nassim Taleb, and a much shorter and more whimsical one by Jacques Derrida.
From Nassim Nicholas Taleb’s The Black Swan: The Impact of the Highly Improbable (2007):
The writer Umberto Eco belongs to that small class of scholars who are encyclopedic, insightful, and nondull. He is the owner of a large personal library (containing thirty thousand books), and separates visitors into two categories: those who react with ‘Wow! Signore professore dottore Eco, what a library you have! How many of these books have you read?’ and the others—a very small minority—who get the point that a private library is not an ego-boosting appendage but a research tool. Read books are far less valuable than unread ones. The library should contain as much of what you do not know as your financial means, mortgage rates, and the currently tight real-estate market allows you to put there. You will accumulate more knowledge and more books as you grow older, and the growing number of unread books on the shelves will look at you menacingly. Indeed, the more you know, the larger the rows of unread books. Let us call this collection of unread books an antilibrary.
Jacques Derrida, during an interview in his personal library for the movie Derrida (2002):
Amy Kofman: Have you read all the books in here?
Derrida: No, only four of them. But I read those very, very carefully.
More quotes and references in the comments are welcome!