Tuesday, January 14, 2014

Everything You Never Wanted to Know: eveapi

This is going to be slightly misleading.  The purpose of this post is to showcase Entity's eveapi Python module, more than the official EVE API.  The ins-and-outs of large scale EVE API work require much more space than I have here, and I am not completely familiar with all the traps and holes.  Instead, this is meant as a first-pass guide to the API and how to leverage it in your custom code.  Also, this will only be about "v2" read-only options, since CREST is something else entirely and mostly a pipedream still.

EVE API Basics

The EVE API provides a read-only portal for apps to get game data.  This can be anything from skill plans, to industrial jobs, to wallet transactions, and more.  The API is accessed using a keyID/vCode combo, which is controlled through your account management page.  There, any feed can be enabled, or disabled, even the key can be set to expire or deleted entirely.  This gives account owners the means to have many APIs for any particular app or service.

A query looks like:


EVE's API returns follow the XML DOM structure.  This is a parsable tree that many code languages have easy methods to handle.  Javascript and Perl are my two personal favorites... Python's XML handler is kinda terrible.

Feeds

There are 4 kinds of feeds.  Account, Character, Corporation, and generic.  Each group has a use scheme, authentication requirements, and similar behavior.

Account Feeds

Account feeds are used to get general information about the key and do not require any special access.  They take a given keyID/vCode combo and return information about that key.  APIKeyInfo will tell you what type of key (Corporation or Character) and what feeds it can access.  Characters will tell you what characters are accessible by the account/key and general info like corporation name and characterID.  These feeds are meant to be used as a validation step.  If you are writing your own app, it's a good idea to validate against these resources, so you can handle errors such as API expiration or invalid key more gracefully.

Character Feeds

I won't break down every feed one-by-one, but Character Feeds are meant to give individual character data.  This can be troublesome if you are given a all-characters key, since the character list will need to be pulled from the Account/Characters feed.  

All of the Character feeds require a 3rd piece of the key, CharacterID.  So, if you wanted all the character sheets of one account, you would have to ask up to 3 times: one for each CharacterID.  

Corporation Feeds

Like Character Feeds, Corporation Feeds require a specific API to access.  These API keys can only be generated by CEO/directors.  The access/information is completely separated from Character Feeds: one cannot access the other at all.

Corporation feeds say they need CharacterID in the documentation, but I believe most don't need it.  Also, the Starbase Detail requires itemID of the tower in question (found in the Starbase List).  As always, check documentation for feed specifics.

Generic Feeds

All other API feeds fall under a generic category and do not even require a keyID/vCode combo.  If you're looking for map info, or a characterID name conversion, or some basic stats, they can be accessed directly without a special key.  Also, cache and limits to these APIs tend to be much more generous.

Accessing Feeds

Authentication is a two factor process.  First, a key must be validated: make sure it's valid, the expected type, and not expired.  Second, to access a feed, it must have the valid "accessMask".  If one is not valid, the API will reject your request.

Every feed has a binary access mask.  To check if you have the right key for the feed in question:

if(API_accessMask & _feed_accessMask_) == _feed_access_mask_: 
    pass 
else:    #API will return HTTP:503 error
Using this snippet means you can access any feed the key gives access to, rather than hardcoding for a specific one-size-only key.  Seriously... I want to crush hands when a tool won't take a 'everything enabled' key

Best Practices

  • Follow cachedUntil guidelines
    • Should re-return same data until cachedUntil expires
    • Can lead to bans if ignored
  • Don't forget User-Agent in your request header
    • Gives admins a way to contact developers who are causing problems
  • Use gzip encoding in your request
  • Check the return header for more info

Using eveapi

Go crawl through some of the API documentation and look at the different returns each feed has.  Though there are some constant themes, each feed has completely different return structures.  If you were to write a new API tool from scratch, you could be faced with a feed-by-feed custom cruncher.  Instead of writing 70-80 custom functions, eveapi lets you get straight to the meat of each feed with a simple set of calls.

How does eveapi work?  It utilizes Python's magic methods to build classes and objects dynamically as they are called.  Though there is no specific auth.char.CharacterSheet() entry in the codebase, it builds the query on-the-fly from the names of each call.  There is a level of elegance and future-proofing that makes eveapi absolutely incredible.

Installing

It's meant as an importable module.  It's pretty easy to copy down a version from github.  Though, to keep a version in your repository current with its current repository, use git submodule.
From there, it's a simple import eveapi from eveapi, and you're ready to rock!  Since eveapi is just a Python module, it makes it very easy to include with a project for deployment where you may not have control of the Python version, like AppEngine.

Working with eveapi

eveapi comes with a tutorial program to show how some of its features work.  Here's a little more ELI5 approach:

1: Create an auth object for the key in question

Here we have 2 objects:
  • auth is a key-specific access token.  Used for any direct Character or Corporation feed access
  • api is the global api token.  Used for generic queries 

2: Validate the key credentials

Though an exception will be thrown when you try to access something you shouldn't, it's best to avoid the exception entirely.  With this code, we pull down the keyInfo, fetch the character list (which can also come from Characters), and validate that we can ask for the Character Sheet.  We could further validate that the account is even active with AccountStatus, but that requires more access than may be given.

3: Pull down relevant information



The art of eveapi is that it accepts the names straight from the <rowset> or any other tag and lets you crawl through with english instead of:
 ...getElementsByTagName(...)[0].firstChild.nodeValue 
--OR--
 for row in rowsets[0].getElementsByTagName("row"): 
   row.getAttribute(...)
Each feed can be accessed by name.  Just combine the /type/APIname without the .xml.aspx

The contents can be accessed by name.  If it's a list, check the <rowset name="[this one]">, if it's a text item, just reference the name.  It's deceptively simple!


Instead of getting hung up in the actual structure of each individual API, each element can be accessed directly, regardless if it's text/attribute/tag/etc.  This leaves more time for contributors to work on building real tools rather than becoming mired in feed-by-feed minutia.

This is by no means an exhaustive guide for EVE API or the eveapi module.  But this should cover the basics for those who may have been overwhelmed by the API feeds previously.