iso5963: guiding indexers (sections 1.3 and 1.4)

In ISO 5963-1985, sections 1.3 and 1.4 define the scope of the document in a couple of important ways. Mostly, that it’s designed to help indexers index documents in ways that are helpful for users. It helps with this by providing a consistent set of guidelines for analysis that indexers use to promote useful indexes inside organizations and between organizations that exchange indexes. (A note from before that this document specifically deals with humans doing indexing, and not algorithmic indexing done by computers.)

So, the goal of the document is consistent subject indexing through making a guide to the document analysis and concept identification stages of the indexing process. To what degree is consistency possible, likely, or desirable?

Certainly some consistency is useful. As users of this index are (presumably) part of a domain or community of practice that actually exists out in the world somewhere, they probably have a common shared vocabulary that they use to describe things and the index should reflect that as much as possible. But, if you interchange between different groups of people, they will probably have different vocabularies for the same (or similar) things, and parts of documents analyzed may be more or less important, causing the subject of the document to change (with regard to the other groups.)

Depending on the type and scope of documents and the vocabulary used, indexer consistency may be quite low. In some ways, this mirrors the usual problems of recall versus precision when trying to retrieve information from a system. If the indexer has a relatively small set of terms that they’re choosing from, or is only trying to cover things in the broadest of terms, then it is easier to come up with common terms than if they’re creating their own terms on the fly or are trying to be very specific.

It isn’t clear, however, that lack of consistency between indexers is actually a bad thing. As long as the users are well-supported in their searches, which is the point of this exercise, why does it matter if the results are nonstandard. In section 1.4, they indicate that they’re specifically trying to standardize practice rather than results.

This entry is part of my ongoing blog entry series on specs and standards, done largely so I have reference to my thoughts later on. It’s also put up with the expectation that it will be helpful to other people. You’re welcome to comment on this, and I may make new versions of this document later that incorporate other remarks or just reflect my changing understanding. One particular note is that I won’t send you a copy of this spec — you should buy it or get it from your local (university) library.