classification

InfoCamp 2007 was a great success, and I am really happy about the way that it went. There were roughly 50 sessions over 2 days, and roughly 85 participants. It was a great relief, and everyone seemed hopeful that there’d be another one next year and there wasn’t too much negative feedback, and that’s about as high praise as you can expect.

My session was called “Thesaurus, Ontology, and Inference” and was about the benefits you get from having a minimal amount of semantic data associated with documents — mostly exposing metadata that’s already present in the system and some things you can do with it. I think my presentation confused a bunch of people because I had to cut so much information out (my session was compressed from 60 minutes to 10 for various reasons), but we’ll see how it plays in the long run. I think there’s a lot of difference in the assumptions that I make based on previous work experience and what the IAs/IxDs in the audience have from their work experience. At any rate, got some decent feedback for refining the presentation.

I made it to about half the sessions I wanted, keeping folks moving in the right direction was a time-consuming project. Low time between interrupts, but a lot of good fun. It was especially fun talking to an audience and getting people to introduce their sessions, and I also had a lot of valuable hallway conversations with people. One particular thing I found was a good venue for publishing professional work other than the one I have already, and the fact that there’s a difference in focus between the two helps, so what isn’t wanted by one may be by another.

All in all, I really enjoyed being one of five people running this conference, it was a great time with a great group of people… I look forward to this community growing.

Indexing schemes are largely similar in their outputs – the right answer to many questions, such as ‘controlled vs. non-controlled,’ how many terms to use, and how expressive or deep the vocabulary should be are all matters of indexing policy considered on a per-collection basis and more or less well-defined in the specification (ISO, 1985). The biggest difference in indexing methods is whether they have an objective or subjective view of the world. Much of the difference between methods after that comes down to how objectively or subjectively they view the world, and what the right approach to dealing with that is.

The most subjective approach to indexing is ‘social indexing,’ which makes no assumptions about underlying reality beneath the words it uses for indexing, and in fact many of the original proponents of it as a method felt that this was a benefit, because each user may mean something different by tags (Guy & Tonkin, January 2006). Folksonomy proponents are backing down from this extreme approach over time, but denying a possible shared reality is as far towards subjective as possible.
A large distinction is between document-oriented and request-oriented indexing, with document-oriented indexing using the properties of the document to describe the document and request-oriented indexing considering what the likely requests of the user community are likely to be (Fidel, 1994). Another formulation of request-oriented indexing is tying indexing to supporting the information-seeking behavior of the users; indexing in this sense is about making it possible for users to find documents that satisfy their information needs (Hjørland, 1997).

Document-oriented indexing and request-oriented indexing are present in all collections to various extent. In general, the more general the collection (the broader the scope of the collection and of the population the collection serves), the more oriented on the properties of the document itself the indexer has to be. This is because the broader the scope of the collection, the less adapted the indexing can be to particular topics. Both the depth of indexing and specificity of meaning of terms have to be limited in a more general collection. (Hjørland, 1997) For a collection more limited in scope, more attention can be paid to supporting the sorts of requests that will be coming in from a user community. (Fidel, 1994; Swift, Winn, & Bramer, 1979) Document-oriented indexing is more objective than request-oriented indexing, because it presupposes (or at least pretends) that there is a single reality that the documents are being described in relation to. Request-oriented indexing is more subjective, because it admits the concept of communities of practice (or discourse communities) with their own conceptions of the world (Hjørland, 1997; Mai, 2001).

The process of indexing is in fact more subjective than this, because the subjectivity of the indexer must be taking into account – there is a series of interpretations that the indexer makes in moving from text to a representation in the subject index(es) (Mai, 2001). The indexers’ interpretation can be informed by the domain, which is called domain-centered indexing. When analyzing a document, domain-centered indexing starts with an analysis of the domain and then moves on to the users’ needs (and indexer interpretation) and then role of the document with regards to these (Mai, 2005).

Indexing approaches vary in a number of ways, but one of the most important is how subjective their view of the world is, and how granular that subjectivity is. Document-centered indexing is least granular (most objective), with a single representation claiming to serve all needs; after that is domain indexing, which considers documents in light of their role in a domain (although other information is taken into effect.) User (or request)-oriented indexing considers the information needs of the users as primary, assuming that the users of the catalog have specific purposes in mind for the catalog. Democratic indexing goes further in assuming the fragmentation of reality, assuming that there isn’t consensus in meaning among users, except possibly in aggregate.

For indexing practice, which of these methods should be considered is tied to the collection. The more focused a collection on particular users or domains, the more tied to requests or domain analysis the collection can be. The more general the collection, the more likely it is to need an assertion that there is a single objective view of reality (document-centered) or that there isn’t a meaningful consensus (social-indexing) – and the decision between document-centered and social-indexing may be more usefully made on other means, like whether the collection is developed (like a library or a set of resources chosen for a purpose) or arbitrary (like pages taken off the web.)

bibliography

• Fidel, R. (1994). User-Centered Indexing. Journal of the American Society for Information Science, 45 (8), 572-576.
• Guy, M., & Tonkin, E. (January 2006). Folksonomies: Tidying up Tags? D-Lib Magazine, 12(1).
• Hjørland, B. (1997). Chapter 2: Subject Searching and Subject Representation Data. In Information Seeking and Subject Representation: An Activity-Theoretical Approach to Information Science (pp. 11-37). Westport, CT: Greenwood.
• ISO. (1985). 5963: Documentation — Methods for Examining Documents, Determining their Subjects and Selecting Indexing Terms. (No. ISO 5963-1985): International Organization for Standardization.
• Mai, J.-E. (2001). Semiotics and Indexing: An Analysis of the Subject Indexing Process. Journal of Documentation, 57(5), 591-622.
• Mai, J.-E. (2005). Analysis in Indexing: Document and Domain Centered Approaches. Information Processing and Management, 41(3), 599-611.
• Swift, D. F., Winn, V. A., & Bramer, D. A. (1979). A Sociological Approach to the Design of Information Systems. Journal of the American Society for Information Science, 30, 215-223.

Technorati Tags: domainanalysis, folksonomy, indexing, indexingprocess, subjective

ISO5963 (ISO, 1985) states indexing ‘extracts concepts from documents by a process of intellectual analysis.’ Any sort of intellectual analysis involves context – at least the context the indexer personally operates in. Hjørland (Hjørland, 1992) distinguishes between ‘content-oriented indexing’ and ‘request-oriented indexing.’ This division also describes what additional contexts are important to the indexing process.

‘Request-oriented indexing’ means indexing documents in a collection in relation to requests that will be made against that collection – using the terminology of whatever field is the object of study. Request-oriented indexing has a high return on investment when the purpose to which the collection to be put and the user population is well-known – both the context of the collection and the user population can be used to develop indexing, both for particular documents and the overall collection policy. Using large amounts of context in this manner for subject indexing would seem to be the clear winner, but it fails to work generally.

For request-oriented indexing to work well, there are several suppositions. First, is that the purpose of the collection remains stable over time. This may fail for several reasons, such as the gradual evolution of a field or members of a different field using the collection (ISO, 1985). Second, that the terminology and object of study of a field remains constant over the lifespan of a collection; but, most fields change over time and it is generally hard to know when creating something how long something will last. Third, the supposition that there is a single shared useful context shared among all the users of a document that will make retrieval in the index system possible. Although it would be possible to index documents within many possible contexts, it becomes an exponential problem as the number of contexts and documents grows (Hjørland, 1992).

Subject indexing is a representation of a document; the purpose for that representation is finding and using that document at some future time – ‘search for items with potential’ for various purposes (problem solving, meeting information needs, gaining general subject understanding) in Hjørland’s formulation (Hjørland, 1997). If the document representations aren’t effective, the searching based on the representation will be of low quality (Mai, 2000). Considering searching as the goal, subject indexing can be analyzed in terms of searching’s metrics, recall and precision. When the indexer is able to focus specifically on analyzing the user’s domain, the user can have high precision and recall. However, as the indexer’s analysis of the documents in a collection moves out of alignment with the user’s context, either precision or recall will drop. Recall drops when the indexer chooses terms that are different than terms the user chooses for searching on a particular subject. Precision drops when the user and indexer categorize the subjects differently.

What determines the right amount of context to use? With a more general collection, the domain will be less specific and the collection less purposively unified and the indexer must make less use of context, leaning more towards a document-centered approach to serve the searches of a broader audience. More specific collections, however, can use a more domain-oriented approach to indexing more aligned to the context that the documents exist in to better serve their specific users.

Bibliography

Hjørland, B. (1992). The Concept of “Subject” in Information Science. Journal of Documentation, 48(2), 172-200.

Hjørland, B. (1997). Chapter 2: Subject Searching and Subject Representation Data. In Information Seeking and Subject Representation: An Activity-Theoretical Approach to Information Science (pp. 11-37). Westport, CT: Greenwood.

ISO. (1985). Documentation — Methods for Examining Documents, Determining their Subjects and Selecting Indexing Terms. (No. ISO 5963-1985): International Organization for Standardization.

Mai, J.-E. (2000). Deconstructing the Indexing Process. Advances in Librarianship, 23, 269-298.

So, just as a helpful hint for people doing Taxonomy type stuff, or whatever controlled vocabulary: The difference between pre-coordinate and post-coordinate terms are pretty obvious when you can actually remember them, but here’s a helpful hint:

Precoordinate: concepts are combined into terms before the thesaurus is created.
Postcoordinate: concepts are combined into terms after the thesaurus is created — ie: usually at time of use.

I know that someday someone who needs this information will find it searching the web, and that makes me (relatively) happy.