Archive News #2 - Tags

[francescacoppa.livejournal.com]

Welcome to our second Archive news post! These regular posts are a venue for us to answer some frequently asked questions about the Archive of Our Own.

Please leave your questions about the Archive in comments and we’ll answer them in upcoming posts. (This is a space for more general questions – if you have specific comments about the design or usability of the Archive please send feedback on the Archive site itself, so it goes into our bugfix and design process).


This week we're looking at something quite specific: the way tags are used on the Archive. This is a bit more detailed than a lot of the posts we plan to have in this slot, but as tags work a bit differently in the Archive than on other sites you may use and we've had a lot of questions on them, we thought we'd do a special feature.

Where are tags used on the Archive?

Tags can be used on works (ie your fics) and on bookmarks. For the rest of this post, we'll mainly be discussing the way tags work on works (bookmarks are a bit simpler).

For the purposes of the archive, almost all the metadata on works - that is Category, Warnings, Rating, Fandom, Characters, Pairings, and what is shown as Tags (when you enter them on the work form they're known as 'Freeform tags') - is treated as a tag. So, all the bits in the grey header box on a work are tags:



The advantage of doing it this way is that it should be easier for people to search on any field they wish. If you click on a tag, then you will be shown all the works using that tag. Tags are also used to populate our search filters, which help you drill down through all the available fics.

What format do tags have?

• They can have spaces - so you can put Harry Potter rather than Harry_Potter.


• They can be 42 characters long - we may allow longer tags in the future, but we have to have some limit on the length, or our database might melt.


• They can't have commas - tags are comma separated at input, so if you include a comma, then the database assumes you're making a new tag (apologies to fans of 'At Swim, Two Boys' and other comma-loving fandoms).

What's so special about the Archive tags?

The tags are a bit more organised than you'll find on sites like Delicious. For those of you who are metadata-inclined, we're using a sort of hybrid of folksonomy (user-defined tags) and classification. What this means is:

• In most fields, our users can enter any tag, in exactly the form they want it.


• Behind the scenes, our team of tag wranglers classify and make connections between tags, building a structure which adds extra meaning and helps make tags as useful as possible to all users.

The reason for this is to ensure maximum flexibility for authors, and maximum ease of searching for readers.

What else do tag wranglers do? How do the tag relationships work?

Our tag wranglers work to ensure that tags give our users as much meaningful information as possible, by hooking up related tags and clearing up ambiguities. The full details of how tags work behind the scenes and what tag wranglers do with them are fairly complex. However, some of the most important things are as follows:

* Some tags are marked as 'canonical'. These are the tags that are the most meaningful when viewed alone. So, Dean Winchester/Sam Winchester would be canonical, while Sam/Dean wouldn't. All the other versions are connected to this tag. Only canonical tags are visible in our filters, so you won't see every tag you ever used in there, but since the canonical versions are hooked to all the others the filters still search every possible version of a tag.
* Some tags are 'ambiguous'. These are the tags which could mean more than one thing. For example, the tag Dean could refer to Dean Winchester in Supernatural, Dean Forester in Gilmore Girls, or a whole host of other Deans. We are planning to introduce a special behind the scenes tag category called 'Ambiguity'. In this case, the tag wranglers will mark the tag as ambiguous but also hook it up to all the possible canonical tags it could refer to.
* Tags are given relationships which put them in context. So, character tags John Sheppard and Rodney McKay 'belong' to the fandom tag Stargate Atlantis.

Some tags appear with 'freeform' or 'character' after them. What's that all about?

Each tag belongs to a category - Pairing, Character, Fandom, etc. Tag names must be unique, so if a tag already exists in one category, then when it is used in another category, the Archive will automatically add the name of the category.

For example:
* Buffyfan1 puts 'Buffy the Vampire Slayer' in as a fandom tag. A new tag is created.
* Buffyfan2 puts 'Buffy the Vampire Slayer' in as a character tag. Oh noes - this tag name is taken and can't be reused! So, the Archive changes it to 'Buffy the Vampire Slayer - character'.

Result:
* XanderGirl5 can click on 'Buffy the Vampire Slayer - character' and find all fics which feature Buffy as a character.
* The Archive has unique tag names and so the database does not melt.
* Everyone is happy.

People might use all kinds of crazy tags! Do I need to search for every possible variation of a tag?

Nope. Thanks to the work of our tag wranglers, different tags which mean the same thing are marked as synonymous. So, some authors will mark their fic Harry Potter/Severus Snape, others will put Harry/Severus and still others will put Snarry. Behind the scenes, the tag wranglers hook all these together so the Archive knows that if you search for Snarry you also want fics tagged with the other possible ways of describing the pairing.

My tag has been categorised wrongly! What should I do?

From time to time, there's a chance the tag wranglers will make a mistake when they wrangle your tag. Maybe you only write Death Note fic based on the anime, and yet your fic is coming up when you search for Death Note (manga). Our tag wranglers are only human, and they don't know every fandom in existence, so what's obvious to you might not be clear to them. If you notice a mistake, then please let us know via the Archive Feedback Form.

How can I make sure the tag wranglers don't make mistakes with my tags?

Be generous with your taggings! The more information you put in the easier it is for tag wranglers to make sense of what you meant.

• Give full names (first name, family name) for characters and pairings, unless that's not appropriate for some reason (i.e. 'Angel' from AtS and BtVS just goes by the one name).


• If your fandom could be ambiguous - for example, you write Death Note fic for the anime only, not the manga - then add more detail. You can put Death Note (anime) in the Fandom field, or just add anime in the freeform tags.

What about RPF?

The Archive of Our Own is RPF-friendly! However, RPF is one of the most difficult areas for tag wranglers, since the way people classify their fandoms varies a lot. Do we use fandom name? Network name? What about general groups of people, like “Canadian Actors”, or historical figures?

We've come up with a concept for dealing with this, but we haven't finished building it, so please bear with us.

What about crossovers?

We love crossovers too! However, we don't have specific tags for crossovers, since crossovers are just fics with more than one fandom.

Please just enter each fandom in your crossover in the Fandom field, separated by commas. Don't separate with slashes, as this creates a tag which we can't wrangle into our search structure. Feel free to add Crossover as a freeform tag, though.

Can I become a tag wrangler?

Yes, probably! We will always need tag wranglers to keep the Archive tags in line. We are looking to add more diverse fandoms to our teams, so while we won't take on everyone who applies (some fandoms are already well-represented) we'd love to hear from anyone who thinks they can help. If you've noticed a fandom languishing unwrangled for a while, it probably means that we have nobody with expertise for it - if you could fix that, please let us know! Anime, manga and comics are currently particularly under-represented.

Some known bugs with tags

We think we've fixed all our known bugs with tags, but there are a few which were extremely noticeable / annoying, so we're listing them here. If you've encountered some other weirdness, do let us know via the feedback form.

• Filtering not specific enough in some cases - e.g. filtering for Stargate Atlantis also brought up fics for SG-1. This was caused by the way tag relationships were working for fandoms with common characters. This should be fixed as of our deploy in late May.


• 'No fandom' showing up as a possible fandom in the filters. This was caused by our need to mark tags which don't belong to a specific fandom (for example, schmoop). It showing up in the filters was a bug, and should be fixed as of our deploy in late May.

We hope this post answers a few of your questions! Please leave other questions and comments here. We won't be answering comments on this post directly - we'll put your feedback into our pool of things to answer in future posts.



Mirrored from original post on the OTW blog.

Comments

[alpha-orionis-v.livejournal.com]

I will be waiting very patiently for development on the crossover and RPF areas, since I've got two fics that would probably fit in both categories and I've no idea how to upload them right now XD

[zooey-glass04.livejournal.com]

You can add RPF and crossover fics right away if you wish! With the crossover, just put the names of both fandoms in, separated by commas. For the RPF, you can classify the fandom however you wish - eg we have some people who have already uploaded Jared/Jensen fics and classified them as 'Supernatural RPF'. We'll always allow people to tag their fic as they wish - the developments will be made in how we deal with tags behind the scenes. So, you could post something like this:

Fandom: Fallout Boy RPS, CW RPS
Pairing: Jared Padalecki/Pete Wentz

Then our tag wranglers would have a headache for how to classify those tags behind the scenes, and would beg the coders to write new code to help them! But you should be able to post away happily.

Does that make sense?

~ Zooey (OTW staffer)

[alpha-orionis-v.livejournal.com]

Well, half of it is RPF of Saturday Night Live actors (and David Letterman. Don't ask lol), and the other half is a movie. I'm just massively confused how to categorise that.

[zooey-glass04.livejournal.com]

Heee, I love fandom.

I'd say put

Fandom: Saturday Night Live RPS, Movie Title
Pairings: Whatever pairings apply
Freeform tags: Crossover

Then anyone searching for either fandom, or the specific pairings, or crossovers in general, will find it. Our tag wranglers will have the rest of the headache!

[alpha-orionis-v.livejournal.com]

Cool. I asked about this a few months ago, and got a big shrug, so it seems like things have been worked on in the interim. ;)

Thanks! :D

[few.livejournal.com]

In that case, your fandoms would be something like SNL RPF, Whatever the Movie Title Is. The headaches will still belong to our tag wranglers!

As a user, you just need to list the fandoms you think apply to your work and let us worry about the rest. We're trying to save the headaches for the people who volunteered, honest. :)

[ruuger]

Just wondering how are you planning on organising the fandom tags for Books & literature? Right now it seems that (in the 'Fandoms' page) some fandoms are called by their series name, some by book name, some by "Book name (Novel)", some by "Book name (Book)", some by "Author - Book"...

How, for example, should I tag a fic based on Ian Fleming's Bond novels (to separate it from the movie fics)?

[elz.livejournal.com]

I'd say "James Bond - Fleming". It's an area where the standard is still evolving a bit, I think, because different book fandoms are just set up in so many different ways - sometimes they revolve around an individual author, sometimes a series, sometimes a character, sometimes a novel, and sometimes a shared universe with multiple authors, and we want to make it as easy as possible for people to find any or all of those. As other folks say above, the guideline for users is just generally "put in what feels right to you, leave the headaches for the tag wranglers." :)

[forodwaith.livejournal.com]

As a librarian, I think I love you for trying to find a balance between controlled vocabulary and free language indexing.

[zooey-glass04.livejournal.com]

Heee! It has been a challenge, definitely! Have you seen PennTags (tags.library.upenn.edu/) - it's not at all suitable for what we're doing with the Archive, but it's quite a cool way of dealing with it from the library perspective.

[cidercupcakes.livejournal.com]

How should I go about dealing with threesomes when I'm uploading fic? In the past, repeated attempts have just resulted in me being faced with errors telling me "Pairings is invalid". Usually my stopgap is to just fill all three characters in at the "Characters" field, put, say, "Character A/Character B, Character B/Character C" in the "Pairings" field, and then tag it with "Threesome", but this seems like a somewhat messy solution. Whether I try with "Category: Multi" or "Category: Other" doesn't seem to make a differences (all of the ones I've tried it with have been mixed-gender threesomes, so simply selecting the M/M, F/F, or Het categories doesn't seem appropriate, and attempting to choose "Category: Het", then "Pairings: A/B/C" returns the same error message as before anyway). Is there something very obvious that I'm missing?

(Alternately, is this something to put in the Feedback form at the site, rather than asking about here?)

[zooey-glass04.livejournal.com]

Hi!

I'm afraid you have run into a legitimate bug /0\ If tags are too long they they error (with that less than helpful error message) - currently a single tag can't be longer than 42 characters. We're working on some fixes for this, but in the meantime tagging your threesomes using characters' first names only will get around it (and then use full names in the characters field).

I hope that helps - it's not you, it's us!

(This would have been appropriate for the feedback form, which is currently also doubling as support, but we're happy to answer questions here too!)

-Zooey
OTW coder & AD&T committee member

[poilass.livejournal.com]

I know this post is a bit old by now, but I am still bewildered by the whole concept of tag wrangling. The organisation system is dozens of people running around behind the scenes, fixing every tag? I feel a bit like you've said you plan to hand-code the html on every story, or told me my tv is actually run by pixies holding up one picture after another, really fast! Is that scaleable technology? How many wranglers do you have, how many do you need, and do you expect to consistently have enough, once the archive has thousands of fandoms?

Will the promise of other people being able to use the software for their own archives actually happen? Obviously you can stick the "open source" sticker on it but if it takes 50 people to maintain it, it's not useable.

I really hope you continue with these info posts (even if you don't get a lot of comments!), and if you haven't got your next one planned already, I'd like to know more about what the archive is doing re: accessibility, and also how the search/browse function is going to be fixed.

[zooey-glass04.livejournal.com]

We're always happy to hear from people! Thanks for your questions.

We currently have about 12 active tag wranglers (there are more people on our books, but not all of them wrangle on a regular basis). They are keeping up with the workload fairly well, although there are certain fandoms which are under-represented. We don't anticipate ever really needing a bigger team than this, and in fact long term we will probably not need so many people. Once a fandom is wrangled it's wrangled, so after the first big burst of new fandoms, it will be more of a steady trickle. Similarly, we'll wrangle certain freeform tags to match synonyms, etc, but as time goes on we'll have covered the main bases. (We're also reviewing the way we deal with freeform tags as part of our search and browse redesign, so we'll be looking for ways to reduce the potential workload there.)

In order to make our tag wrangling work effectively, we're actually going to be preloading the Archive with a lot of tags (rather than waiting for people to add them on their stories). This means that we'll have a big burst of work at the beginning, but then those things will be in place for the future.

It's important to note that even without any wrangling, a lot of the Archive will function quite happily. The code is set up so tags are recognised as having been entered in the Fandom field, for example, so to some extent they are pre-wrangled. Our Search function is very comprehensive, too, and will find things whether they are wrangled or not - the wrangling really makes it all shinier and more interconnected, but the underlying search possibilities are already there.

In terms of offering the software as open-source - yes, this is definitely still very much the plan! For people who are interested in running a multi-fandom archive, we will offer our own files of pre-wrangled tags to populate their databases. We'll also be looking at other ways to ensure that the software is functional for a variety of different needs, including a tiny single-fandom archive with one mostly absent mod. Although we do try to keep future use in mind while we code and design, however, we won't be concentrating on that side of things until after we have all the code done for our own needs. But it's not forgotten about, and we are aware of the fact that not everyone will have the same setup as us.

We have a topic for the next news post, but thank you for the suggestions for future topics, which we'll schedule for another post in the near future. In the meantime, just to keep you personally updated, I can say that we are currently undertaking an accessibility review, although if you notice specific issues please do log them via the feedback form on the Archive. Because we're making a lot of changes to design and code, new accessibility problems tend to crop up despite our best efforts. Improving our accessibility is a main aim, though. Regarding search and browse, thanks for your feedback submitted via the feedback form, and we are intending to make the kind of improvements you request, but it will take time because it's a big piece of design/coding, and a really key one which we want to get right.

Thanks for your questions, I hope these answers clear things up a bit for you. It's great to know you're finding these posts useful!

~ Zooey, OTW AD&T Committee member and Coder

[poilass.livejournal.com]

Wow, thanks for such a detailed response. I am much reassured ;}.
After I sent my cranky, longwinded feedback re:search (could've just said: "OR" function pls! doh,) I saw that plans were afoot to make some changes, but I wasn't sure what they included. Thanks for letting me know!