My recent article about price feeds being broken in Odyssey 1.1 was picked up by Cap Stable as their "Big Talk" segment (~30min in). I'd like to offer a few points in friendly rebuttal.
The Scope of Impact
The hosts state that the feeds were completely removed, and this is overstated. As I mentioned in my article, the namespace was altered, but the actual data remained intact. This broke the "scrape" step as featured in the flowchart on the article:
.png)
In essence, this left the entire rest of the price aggregation network idle, because its expected feeds were silenced. The hosts are correct in their assumption: if the cache is encrypted or removed, this entire network shuts down. But the reality is the codewords inside the cache were changed, not the actual data. Also, CCP was quick to put out note that the problem was unintended. And thankfully, by the time my article was published on TMDC, EVEmon had already rolled out an update returning the expected feeds to aggregation.
Darius Johnson, formerly known as CCP Sreegs, is pretty infamous for his hatred for this relationship. I can't rebut that, and he called me out on the comment thread on TMDC, and it's worth reading. The TL;DR, from my perspective, is that we're stuck in a crappy confluence. On the one hand, there is a working system, and infrastructure, that enables not just the market (a pivotal pillar of the EVE Online feature set, IMO) but a whole myriad of other 3rd party features. On the other hand, this entire ecosystem is outside CCP's control, and therefore enables a lot of less-than-reputable and unintended clients (though I would argue, far less than you'd think). Then, to propose a fix, you're coming to a losing battle:
- The dev cost to implement is moderate: I'd estimate 2 devs @ 10wks
- The asset cost is non zero: new, dedicated server space. I'd estimate at 10k USD once support is factored in
- The reward for success is low, and the goal bounds are narrow: only if it works 90% like EMDR will it be counted as successful. Only if both live "firehoses" like EMDR and simpler EVE-Central like feeds are maintained at approximately the same quality (availability and accuracy) as today, will it be hailed as successful.
- The penalty of failure is high: if the cost to update 3rd party tools into a new environment is too high, or the quality of those feeds is too low, the API will go unused. TL;DR: the 3rd party devs will see failure as a direct "fuck you".
I agree that a CCP official feed is needed, and the client cache should be sunset... but CCP needs to deliver a replacement, and half measures won't be adequate. Personally, if market feeds become unusable, my main means to play evaporates. I would probably quit, because the sector of the game I enjoy would evaporate and be mired in too much minutia to be worth pursuing. Also, my faith in CCP is thin, since the last year has supposed to be big for CREST rollout, but the development will behind it has been lackluster.
Better Margins is Better in the End, Right?
This argument is based on a fallacy. ISK isn't the balance beam. The question you should be asking is: Is EVE a better game without those price feeds? Is the universe bigger or smaller because of those feeds? Are we enabling enriching game play?
I'd, of course, argue "yes". The data enables a whole host of cool tools, and makes EVE the game much bigger as a result. Killboard ISK estimates, logistics estimation, shipping, industry... these price feeds give us players a link to the outside world to explain what we do, and why it's interesting.
I'll boil down my argument against the "better margins is better". If margins go up 10%, but effort increases 100%, I am actually losing, and losing big. If margins go down 10% and effort goes down 40-50%, I am winning in the long run. At the end of the day, the ISK is just secondary to the work. If the gameplay is well designed, it will be profitable, regardless of the out of game monkeying.. because I will be limited by the game on how I can participate. The argument I like coming back to is this: I might see a crazy %margin on a T2 product, but the reality is I am volume limited by the build times or invention steps. Just because I CAN make money on a product, doesn't make that the sole metric worth pursuing.
That CCP Guy With the Data
You're thinking of @CCP_Diagoras. And I was a huge fan of the feeds he was giving, but the rumor is he ran afoul of Internal Affairs. Also, the effort in disseminating those feeds, through legal and NDA oversight, I think this isn't coming back... even if they are expanding their data team.
12 comments:
Doesn't the API still produce information about your own market orders? So, the data is there...the API is already there...
People would need to start registering an API with the Market Data clearinghouse to have their orders show up on sites like Eve-Central, like people currently do with the killboard sites.
The big problem for CCP, I think, is the volume of data required to drive these market information sites. For CCP to set up a firehose feed from their systems would be a terrible bandwidth problem. The market cache scrapes are actually taking a big chunk of bandwidth load off of CCP, as the only data being sent by CCP is market data for items that people are interested in buying or selling in regions where people have active clones. (Eve-marketdata web browser tricks aside.)
Perhaps you could get CCP to do a tickertape feed of completed sales by location, price and volume. Or a ticker tape feed of new buy and sell orders by location, price and volume. I suspect you'd need both, really. (Ticker tape meaning that if you missed something it's gone)
How about if CCP creates a full market-data feed API, then they charge 1 Plex (per region?) per month to subscribe to the feed? It'll put some pressure on the market data aggregators to turn around and charge for the data they provide, but the cost is not so much as to break the backs of the sites.
Had a big fancy reply, then my computer ate it!
Killboard like feeds: Though kill data is largely benign, and often replicated, wallet data is far more sensitive. There's nothing technically unfeasible about it (and I imagine this is plan-b if Contribtastic dies). I anticipate the player resistance to be high, and the quality of most data to be low. Would probably work for the highest-volume stuff (manufacturing materials, common mods/ships) but would leave immense blind spots.
Data Volume: The nice thing about EMDR architecture is the actual daily bandwidth from shotgunning data out to listeners is surprisingly low. It would be up to CCP to make a set of market crawlers that generated that data suitably (I'd propose an algo that provides freshness proportional to in-game requests + some offset), but then providing an EMDR endpoint like @Fuzzysteve proposes would be the easy part. Also, by using the existing standard, existing tools remain intact.
Tickertape: This isn't a terrible idea (and I'd expect the crawlers to work on this principle). The problem as I see it is there are some 5-7k items you care to know the price of, there is a sliding scale of how fresh that data needs to be. Tritanium=high priority, officer guns=low priority. I'd rather see something push data from the server as it's requested to a middleman program that then barks that data at the EMDR endpoint.
Charging for Data: This is a slippery slope, and CCP has been trying to fly it periodically with the CREST rollout. Personally, I think players and CCP have a symbiotic relationship, and by rolling out data for free encourages the same from players. As soon as money/PLEX gets involved in it, the relationship changes to "what's it worth to you", and that's a tough question to ask if we're talking about EVE-Central. Frankly, the amount I rely on them, I should probably be footing some of their server bill, but I truly appreciate the free feeds they provide for the betterment of the app environment.
Also, on pay for data: there's a bit of an uneven relationship in previous proposals. If CCP is going to charge for access to the feeds, then players need a means to recoup that cost. Some iterations of the 3rd party EULA have been incredibly one-sided in this bargain.
The EMDR people may know better the amount of bandwidth required to supply market data out to sites. The issue here is a good example of Business Intelligence.
There are two qualities for good Business Intelligence, Accuracy and Freshness. The current system is actually poor in both areas. Relying on dispersed and erratic data gathering results in chunky and inaccurate data.
Ideally, a market data web site would have full access to the market data directly in the Eve server databases. It would then be on the web site to process and present that data in a format usable by the people that need the data. However, CCP hosting market data on their own web servers is the only way this would happen.
You're suggestion that Eve supply a new server to host market data would be second most ideal. They would have to supply a server that can be queried fairly regularly and they dump data into the box fairly regularly. The quicker they dump data into the box, the fresher it will be. Accuracy shouldn't be a problem in this case. The relay sites that are around now could still do what they're doing if the query cycle from the BI box was time or bandwidth limited in some way.
A transactional data feed would be a third best option. This sort of API would require third party sites to collect the data and relay it. This is closer the current system, with greater accuracy but perhaps less freshness.
Word on the street, or perhaps back rooms in this case, have it that Diagoras was shown the door. Was made to look like he jumped, but some people are saying he got a little push.
So the large traders/industrialists running tons of sophisticated/automated 3rd part market tools are wondering if their gravy train is over, and they will have to do like most players do, and drag most info into spreadsheets by hand, or at least with less powerful tools.
Yeah, I feel terrible about this. Suddenly Eve sounds more democratic.
What was undemocratic about tools that were available to everyone to begin with?
Doing industry is a colossal pain in the ass even with our tools. If that data went away, the effort to reward ratio would plummet to an unacceptable level.
Like I said, the rumor I've heard was he got the stick from IA for actions completely disconnected from his twitter feed. But it's only been back door murmurs, I have no actual source on it.
Like I said, the rumor I've heard was he got the stick from IA for actions completely disconnected from his twitter feed. But it's only been back door murmurs, I have no actual source on it.
@ Kelleris.
I used to run the industrial wing for my null sec corp when we lived in Pure Blind before the fall of NC. I also did a lot of industry in my own high sec corp later on, with a few buddies from that old corp. We had no issues doing it by hand, with our own spreadsheets.
Was it as effective as going to Fuzzworks, or half a dozen other sites, to do a quick lookup as to what was the most ISK/ mfg hour? No. of course not. But if everyone is working from the same level playing field (ie. no automation), then the mini-game of industry is fair for everyone.
And as null sec players are so fond of saying, adapt. Some industrialists will drop out of the race, and that means higher margins for those that adapt the best.
It is all academic anyway. CCP will cave to the screams, and will prioritize a fix for this, if they have not already.
1) automating price fetching isn't difficult. The feeds are free and easy to implement into anything that can parse it (gdoc, excel, etc). You're not even limited to a single source.
2) automation is only acceptable as a READ operation. Actual market bots that update faster/longer than humans are definitely something to avoid, but are a separate problem to be tackled independently.
Frankly, if you're insisting on doing it the hard way, I can't stop you, but it's not like you're specifically out of the loop on making decent tools. I personally make almost all my spreadsheet work open source here and you're free to take from my tutorials and code snippets if you're not code oriented. Otherwise, it's complaining because someone brought T2 to your T1 fight: adapt or die.
Post a Comment