CFC Interdiction Follow-Up

This was an interesting project.  Thankfully, some recent tools made the work go much smoother, and I'm really starting to sink my teeth into SAS JMP.  This piece was a lot of fun to write, since the overall theme from the graphs is "CFC failed to make sustained impact to the market", and TMDC is so widely regarded as only a propoganda hole.

• CFC Caldari Ice Interdiction: Final Impacts
• Imgur Graphpr0n gallery
• Ice Raw data
• Secondary raw data
• Mining raw data

Cutting Room Floor: Mining Report

This has been a topic I've wanted to cover, but haven't had the time to really cover it.  Also, the story is pretty brief.  

The story I wanted to tell was "ice miners, when harassed, will switch products", but the interdiction was not strong enough to force that to happen.  The story instead is a slow and steady decay in mineral prices.  It's interesting to see ice overtake minerals again, but I am not completely sure of the reasons.  When I tried to expand the story, I felt that the graphs were "show"ing more than I could really "tell".

Personally, I've been blaming the anemic market on the end-of-summer doldrums.  Unfortunately, I didn't know how how extensive the lull was.  

Graph Credit Ripard Teg at Jester's Trek

In my opinion, I think this is the most blunt graph to showcase on the state of EVE as a whole.  With no major conflict (CFC v TEST in Fountain drove the post-Odyssey spike) and a general malaise in accepting the status quo, my predicted rebound will probably fall far short of my personal expectations.

If I were to speculate in larger terms, I'd say that the slow slump in mineral prices is due to supply remaining constant while demand shrinks.  I could even further speculate that though the number of participants is declining, that there is some supply floor furnished by bots.  I admire CCP's fight against bots, but in a world where eradication is impractical, you are forced to take them into account at some level.  Unfortunately, I have no way to track bot numbers from any of my data feeds, so this effect is 100% speculative.  If demand picks up, I expect the price to flatten out and slowly rise, since full time mining is boring and not as lucrative as other endeavors, but unless player traffic drastically improves, I expect these swings to be very slow.

It's important to remember that we all exist in a circle of life between PVP and industry.  If PVP isn't destroying, industry has no one to sell to; if PVP outpaces industry, the price to PVP goes too high, and people become risk adverse.  My hopes are as we go into the winter, and through Rubicon, that moon incomes will recover to a level that CFC space starts to look enough to start prodding.  Also, I wonder if CFC leadership is intentionally spreading the forces thin just to test what the real end of their reach is.  

Graph Magic

I wanted to step up the quality of the graphs this time around.  Thanks to JMP, I was able to play with some best-fit tools.  

Using the default "best fit" method (or Smooth) isn't a great fit.  Though it is better than nothing, the fact that graph is so stark between periods, the smooth fit just isn't as useful.  Thankfully to a presentation at work, I looked up Kernel Smoother, which gives a bunch of knobs for how to best fit the data.  By changing the fit function to cosine and lowering the weight to better match the data, I'm able to put much better fit lines.  The best part is I can export the results from my best fit fiddling and move those to graph in any way/shape/form.  You'll see the numbers in the raw dumps.

This was exceptionally useful in my favorite graph of the piece, the Tags4Sec graph.  as a reference, this is kinda how it started:

Volume data is particularly spiky.  Being able to line up the wider trend of peaks and valleys I think gave a much more useful picture of the data rather than overlaying that noisy base data.  When I add the default smooth line, it's pretty clear that line tells the "macro" story, but isn't really useful for this project's analysis.  Also, the "repair rate" was an attempt to combine 3 volumes together in appropriate parts.  In this case, it was 1 part Trainer, 1.5 parts Recruiter, and 1.5 parts Transporter.  In retrospect, I could have been more creative with the combination, since I seriously doubt people were going all the way to -10 in ganking, so equal parts Recruiter/Transporter with no Trainer would have been a better picture.

What's Next?

Economic topics are running pretty lean lately.  There is the noise about PLEX prices skirting 600M and then dropping like a stone, thanks to an Amazon sale.  Also, I've been requested to analyze the "special edition" assets, to try and make a ruling on whether it's best to hold onto it as a collectors item, or dump in the initial release frenzy.  Beyond that, I'm out of topics until Rubicon shows up.

Also, though I am loving how much JMP is enabling me to get these sexy graphs out for people to enjoy, there's a licence problem: I have access to JMP at work, but players aren't going to buy keys.  I'd like to ween myself onto tools like R, but there's a lot of code between where I am today and being able to slice-n-dice as easily in a new set of tools.

Rubicon: Quick and Dirty Speculation

Jumping the bandwagon on this one, I figured it's worth trying some "shoot from the hip" predictions.  I didn't get to watch, so my info is second source.  Going to pinpoint my predictions in T2, I'll let others speculate about the surrounding metas.

The Obvious

Marauders: winners

This isn't exactly news, but some of the features are getting ironed out.  I think the new meta around Bastion are neat, but the hull is still expensive.  I see <40 man gangs adding this to their roster, but not the big nullsec brawls.  They are just too adaptive for the epic-space-battle meta, and their cost is almost 2x that of other standard doctrines.  I am adding the capability to do marauders to my production lines, but I have yet to see the market wind up in excitement.

Interceptors/EAF's: big winners

With interceptors getting bubble immunity and EAF's getting boosted to look like their big-brother counterparts, both these ships will be in MUCH larger rotation.  Pair that with the proposed warp-acceleration change, and I think we're poised to see these ships really come back into the fold.  I worry about the T1/T2 balance in this case, but I am very happy to see more T2 hulls come back into vogue.

Recons: Losers

With EAF's getting such a generous boost, I see recon use dropping off sharply.  Without a buff to effectiveness or EHP, it looks like we'll be seeing even less recons than today.  Simply put, if an EAF costs 1/3rd the cost of a Recon and gives 80-85% of the effectiveness, I can't see the need to put the extra money down for a Recon.  

Dictors: Winners

With the warp-acceleration change, these stand to become much more mobile.  For losing out on interceptors, and gaining that weakness, they stand to become more versatile in leapfrogging your opponent in a chase.  Also, murmurs are that the added warp maneuverability is going to lay some pretty strong blows to capital/supercapital deployment.  

Hictors: Winners?

I want to believe that hictor + cyno jammer = elite supercapital hunting force.  It's rumored to be getting a warp-acceleration boost above its cruiser counterparts and will be far-and-away faster than its prey... but I'm conflicted.  I wonder if the EAF > Recon problem of ISK/utility will be mirrored in Dictors > Hictors.  Hictor volume is pretty critically low at the moment, so I think the only option is to go up from here, but I won't be rushing to the factory to churn a big pile of them.

Not So Obvious

Moon Goo

I'm wishy-washy on this prediction, but the Siphon Unit stands to change the moon goo game.  On one hand, this will wrestle the ability and effectiveness of stockpiling from people trying to artificially constrict supply, applying downward pressure.  It also has the ability to lower net supplies by a small margin, adding some upward pressure to the price of business.  Odds are it's a wash.

The Siphon Unit does look to interrupt any point in the moon goo reaction tree though: extraction, simple reaction, complex reaction.  Which makes the whole picture even more foggy.  It definitely puts the brakes on my moon reaction farming plans, since the bar to harass my operation in low-sec is pretty low.  I keep flip flopping... stay tuned.

Third, with so many T2 hulls getting love between Odyssey 1.1 and Rubicon release, I think we stand to see some interesting shifts in the moon goo market regardless.  As consumption picks back up going into winter, I think this will be the biggest net driver of prices, and I believe that pressure will be up.  I don't think many of the biggest surges and bottlenecks will emerge until January 2014, but between T2 hull consumption increasing and the Siphon Unit we'll see moons come back to being a conflict pivot point.

PI

With POCO coming to high sec, and CONCORD turning a blind-eye to defending those resources, I think this will push the price of PI products up.  With HS carebears being squeezed on getting their low-margin products off planet, this has the opportunity to swing products pretty significantly.  There are also A LOT of planets, and if CONCORD is not intervening, this could be noisy to start and die down when the new stable medium is found.

Remember your First Rule of EVE before speculating on my opinion, but I figured it was a good chance to put some words down to check later.

Nothing Worth Having Comes Easy

There was a troubling sentiment stirred by my recent posts about out of game market feeds being broken.  What's worse, outrage seems to be woefully misdirected on this issue; not so much at the tools but at those that use them.  If you'll pardon a little rambling, I'd like to see if I can get our collective heads on straight about the issue.  This has been distilled down from some far less eloquent frothing rants.

What Is Truly Upsetting

The out of game feeds are furnished by breaking the rules.  Hem and haw over the actual value those tools enable, but the simple and undeniable truth is scraping the cache is against the rules.  Now, CCP is not Judge Dredd, and therefore is treating this issue similarly to US DOJ vs states with marijuana legalization: "technically illegal, but not worth enforcing".  I don't believe we are in danger of market feeds actually being taken away, but I am afraid CCP will never see the effort of internalizing those feeds as worth pursuing.

I think it's important to stay on topic with this issue to make sure we build a request that CCP can reasonably deploy and sunset this weird gray relationship market aggregators live in  Though I think there are opportunities for development that CCP is ignoring, that are enabled by cache scraping, the real truth is the cache needs to be removed and the security hole plugged.  Market feeds are important to 3rd party devs, but an official stream is a better solution in the long run.

Tools are Unfair

This is the sentiment I've found troubling.  That somehow, those who take the time to develop and use tools to increase their effectiveness are "ruining the game".  Frankly, this is a crock of shit.  EVE is hardcore Anarcho-Capitalism at its core.  Those with the largest legion, the deepest pockets, the best tools, and the most charisma are going to win.  You are not entitled to profits and success, you have to work for them.  And even with work, you can lose it all.  It's the nature of the cold terrible game we play.

In my mind, this is tantamount to complaining that guides are unfair.  That somehow doing things "the hard way" rather than looking to build off those that came before you is the purest way to play.  Furthermore, to decry me because my solution is better than yours and you haven't invested the effort to try and beat me... it's just silly.

I am more than happy to explain mechanics or share tools, it's a game and we're a community... but I won't be made the villain purely because I spent (considerable amounts of) my time perfecting my small segment of gameplay and you did not.  I am but one man, and with enough work, time, effort and friends you too could surpass me... I have no secret sauce, only a lot of experience in figuring out what does and does not work.

You Can Only Change Yourself

There are a lot of common whines in most PVE/industry/market circles.  The most common can be summed up: those with no life are making it impossible to play casually.  From "the 0.01ISK game"  to "mined minerals are free" to "incomprehensible market behavior" there is no limit to outrage that certain players are playing the game wrong and ruining it for the rest of us.

The long and short of it is, EVE is a sandbox.  If you don't like a certain segment of play, it is usually within your power to avoid or change it.  

• Hate competing against 0.01ISK changes?
• Move out of Jita/Amarr/Dodixie/Rens
• Operate in lower volumes or in less cut-throat products
• Step up your game: split your orders and beat them at their own game
• Get friends, start your own 24hr cartel!
• Think T1 margins are too thin?
• Don't build T1
• Find miner contractors who think their free time is free
• Become more agile to capitalize on margins as they become available
• Absurd market moves ruining your margin?
• Buy out very low orders
• Raise buy-order floor and snatch up low price goods automatically
• Hire mercs!   Kill them all!

If you change yourself to suit the environment, you will find far more opportunities than waiting for the game to be magically changed to suit your demands.

New Ventures

I have a sickness.  If I am not utilizing a high percentage of my characters, I feel like I'm wasting something.  Right now, I have 12 characters (4 accounts) to use:

• 9 are skilled for T2 production
• 3 are exceptionally skilled for T2 ship/capital production
• 2 are additionally trained for T2 frigate work
• 1 is PVP main
• 2 are "director alts".  Currently idle
• 1 is cyno bitch for main

Currently, my eyes are on 3 characters:  The main, and his "director alts".  Main is already idling his production capacity, and though I'd love to keep him in regular work, it's a little too much dancing to keep that active.  So, I am turning my eyes from traditional production to some more ambitious ventures.

Moon Reaction Farming

I recently built a tool to help track moon reactions like my T2 manufacture sheets.  If you're interested in the actual numbers, check out FuzzySteve's Reaction Tool.  Though the numbers today are rather anemic, the effort for high parallelism doesn't look too terrible.  Though I would adore having some sov space to save on fuel bills, I am SOL on that front.

The Plan is to be able to drop something like 5 large towers (maybe 2 med ones too) and be able to manage reactions on a weekly timescale.  With a more-agile operation, I am hoping to cash in on some of the wider margins as they become available.  Now that I have a jump freighter in my fleet, this makes managing this operation a possibility.  The up-front costs are a bit high, and I may have to stagger the deployment, but there should still be plenty of unclaimed real estate for milking a little more ISK out of some idle character time.

I'm still 50/50 on this plan.  Even trying to stick to an agile reaction system rather than monthly yield has had some bad results in my tracking.  This will only be viable if the margin (300M/wk) is relatively efficient (>20% margin) and low volatility.  In my experiment, I have been watching fullerides, as if I were reacting them, and it's just recently gone upside-down in a rather significant way.  Until T2 production volumes really start to recover, I am uneasy about dropping a bunch of ISK on this.  The fear is, unless T2 is consuming a high volume of component products, then these products will be wild and unstable.

High Value BPO Science Lab

This one is a bit more ambitious, and looks to put my director toons to work.  Something that has always bugged me is it seems to be more valuable (in the long term) to research/copy high end BPOs than actually produce off them.  Also, I have this frustration with Jump Freighters: BPC acquisition is a serious bottleneck.  Either you eat 10-25% of the profit margin buying off contract, or you're stuck building them at such a low rate that it's more a hobby than a revenue stream.  

Still scratching out the details, and grinding up the skills on some very old alts.  It's going to be a long ramp time on this one before it reaches 100%.  The hope is to add 4B/mo in BPOs to the library being researched and rotate them in such a way that every week has yield of some sort.  The final goal is something like 6-8 BPOs of 2 freighters (leaning Obelisk/Charon) in copy rotation, with some 3-5 slots in research rotation on stuff like Moros/Archon.  But, the napkin calculation puts that at ~40B or more.  I had mentioned I needed more long-term goals to justify the ISK I keep grinding.  

The lead time on this is rather spectacular.  To make the copies viable, I will need 2-3mo of ME/PE research, so the results still have value outside Jump Freighter invention.  This acts as insurance in the long run; the added value of research + viable build copies means I can abort and make back my money and then some.  Also, that means any BPO that is added to the collection has the same lead time before productivity, and leaves me pumping ISK into the project for a payoff that's quite a ways out.  I may need to mix real revenue generators into the work in the first 6mo to at least make the effort pay a little.

Why Not Capital Manufacturing?

I am always in awe of eve-fail and K162space that they can thrive industrially with capital ships.  Personally, I find the whole realm high effort and low yield.  By the time you factor in all the hauling, start-up investment, intermediate work, and operating costs, I find the margins not worth the effort.  Add to that the fact that every corporation and alliance seems to have dozens of capital manufacturers, and that customers are concerned with pinching every penny to get into a capital ship, it's not an environment I really like working in.  I am a little intrigued to see if the BPO research venture is a means to leapfrog into supercap building, but that's at least another year out, and I'm in no rush to get there.

It's not a terrible venture for my PVP main to pursue, but the lack of sex appeal means its a distant Plan-C.  I'd rather do something new and exciting than tread the well beaten capital manufacturing path.  It's not a bad path if you have access to a JF and want a use path for an in-house mining program, but seeing as I'm doing this largely alone or in very small joint cooperation, capitals just don't fit my needs.

Jump Freighter Collaboration: Complete

Last night, the last Anshar of the collab between @K162space and myself came out of the oven.  This has been a fun project and a fun experience to play with the more intricate and rewarding realms of industry.

First, a breakdown of the numbers:

Ship	Build Cost	Sale Price	Profit
1	5.325B	6.200B	875.5M
2	5.375B	6.620B	1,245.25M
3	4.867B	6.540B	1,542.55M
4	4.900B	6.351B	1,452.20M
5	5.140B	5.975B	836M
6	5.345B	5.975B	630M

Final profit: 6.600B
Payout Per Contributor: 8,300B (5B buy-in returned + 3.3B profit split)

How Did the Project Go?

I'm willing to definitively state that this project was a success primarily because of some prudent purchasing right before Odyssey released.  My goal was to generate 1B/hull in profit, and in the end we hit that goal, but the last two ships sold for really pathetic margins.  For 28d build time, 630M really is peanuts for the effort/risk/cash involved.  But it definitely beats the Post-Odyssey alternative of near-zero margin

Other than some time setbacks on my end, I think we stuck to the rather aggressive production schedule.  This whole project was lubricated by Blake's generous BP donation, helping to augment my own production assets.  Also, despite the risk with this much cash on the line, things ran surprisingly smoothly.

Also, thanks to this project, I purchased the last ship off the line for my own fleet.  I would have loved to have upgraded the old Obelisk: Heavy Metal Queen, but the production cycles worked out that I had very-low-margin freighters already built, and T1 minerals continue to slide... so I burned those instead.

The reason the above is such a problem is that market price tends to follow the instantaneous build cost, without factoring in the real build times.

As the "closest to reality" chart I could build on short notice, I think the following really illustrates the project reality.  In essence, we were able to produce 4 of the 6 Anshars with the "old" material requirements, and milk that margin.  This isn't 100% true, but imagine the orange line converging into the red line near the end.

What Now?

We are suspending JF production until at least October, if not a little later.  There's a very tight bottleneck on generating BPCs, and producing with these margins is tantamount to throwing our ISK away.  I have some private plans on my own end to try and match Blake's BPC generating efforts, but I don't see those solidifying until the end of the year.  There will be a blog post this week about more industry plans.

Also, as has been ranted about, production margins are pretty poor across the board.  But, this is a function of the time of year more than anything economic.

Graph Courtesy Valkrr 

I have been taking a bit of an EVE holiday myself, running the manufacturing machine only on products that have exceptional margins AND low effort.  I feel bad idling my bread-and-butter builders, but the margins just aren't worth participating at the moment.  Without major conflict or major player traffic, it's not worth the effort to keep the production lines running.

As I said above, we intend to revisit Jump Freighter joint operation again in 4-8 weeks.  No quitting here... just taking it easy in EVE and working through my Steam library.

A World Without EVE-Central

There is a lot of discussion around EVE-Central because, once again, the method behind their feeds is under the public eye.  The honest truth is: Market feeds are critical and enable a myriad of tools, but the method those prices are gathered is illicit.  As much as I will hem and haw about "CCP, don't fix what isn't broken", the truth is the current relationship between market feeds and the client cache is not good... and the relationship only stands to get worse over time.

My recent article about price feeds being broken in Odyssey 1.1 was picked up by Cap Stable as their "Big Talk" segment (~30min in).  I'd like to offer a few points in friendly rebuttal.

The Scope of Impact

The hosts state that the feeds were completely removed, and this is overstated.  As I mentioned in my article, the namespace was altered, but the actual data remained intact.  This broke the "scrape" step as featured in the flowchart on the article:

In essence, this left the entire rest of the price aggregation network idle, because its expected feeds were silenced.  The hosts are correct in their assumption: if the cache is encrypted or removed, this entire network shuts down.  But the reality is the codewords inside the cache were changed, not the actual data.  Also, CCP was quick to put out note that the problem was unintended.  And thankfully, by the time my article was published on TMDC, EVEmon had already rolled out an update returning the expected feeds to aggregation.

Darius Johnson, formerly known as CCP Sreegs, is pretty infamous for his hatred for this relationship.  I can't rebut that, and he called me out on the comment thread on TMDC, and it's worth reading.  The TL;DR, from my perspective, is that we're stuck in a crappy confluence.  On the one hand, there is a working system, and infrastructure, that enables not just the market (a pivotal pillar of the EVE Online feature set, IMO) but a whole myriad of other 3rd party features.  On the other hand, this entire ecosystem is outside CCP's control, and therefore enables a lot of less-than-reputable and unintended clients (though I would argue, far less than you'd think).  Then, to propose a fix, you're coming to a losing battle:

• The dev cost to implement is moderate: I'd estimate 2 devs @ 10wks
• The asset cost is non zero: new, dedicated server space.  I'd estimate at 10k USD once support is factored in
• The reward for success is low, and the goal bounds are narrow: only if it works 90% like EMDR will it be counted as successful.  Only if both live "firehoses" like EMDR and simpler EVE-Central like feeds are maintained at approximately the same quality (availability and accuracy) as today, will it be hailed as successful.  
• The penalty of failure is high: if the cost to update 3rd party tools into a new environment is too high, or the quality of those feeds is too low, the API will go unused.  TL;DR: the 3rd party devs will see failure as a direct "fuck you".

I agree that a CCP official feed is needed, and the client cache should be sunset... but CCP needs to deliver a replacement, and half measures won't be adequate.  Personally, if market feeds become unusable, my main means to play evaporates.  I would probably quit, because the sector of the game I enjoy would evaporate and be mired in too much minutia to be worth pursuing.  Also, my faith in CCP is thin, since the last year has supposed to be big for CREST rollout, but the development will behind it has been lackluster.

Better Margins is Better in the End, Right?

This argument is based on a fallacy.  ISK isn't the balance beam.  The question you should be asking is: Is EVE a better game without those price feeds?  Is the universe bigger or smaller because of those feeds?  Are we enabling enriching game play?

I'd, of course, argue "yes".  The data enables a whole host of cool tools, and makes EVE the game much bigger as a result.  Killboard ISK estimates, logistics estimation, shipping, industry... these price feeds give us players a link to the outside world to explain what we do, and why it's interesting.  

I'll boil down my argument against the "better margins is better".  If margins go up 10%, but effort increases 100%, I am actually losing, and losing big.  If margins go down 10% and effort goes down 40-50%, I am winning in the long run.  At the end of the day, the ISK is just secondary to the work.  If the gameplay is well designed, it will be profitable, regardless of the out of game monkeying.. because I will be limited by the game on how I can participate.  The argument I like coming back to is this: I might see a crazy %margin on a T2 product, but the reality is I am volume limited by the build times or invention steps.  Just because I CAN make money on a product, doesn't make that the sole metric worth pursuing.  

That CCP Guy With the Data

You're thinking of @CCP_Diagoras.  And I was a huge fan of the feeds he was giving, but the rumor is he ran afoul of Internal Affairs.  Also, the effort in disseminating those feeds, through legal and NDA oversight, I think this isn't coming back... even if they are expanding their data team.

Some Graphpr0n Filler

Kind of twiddling my thumbs waiting for the CFC ice interdiction to be called complete.  But, thanks to some productive weekend work, I have been quickly expanding how I can parse/crunch this data and get it out even faster than before.

Thanks to my recent eve-marketdata history scraper, I can quickly generate all the data I care to analyze.  I have a couple plans for some zkb data to pair sales/killed data together, but you'll have to watch my github for progress.

I was under the impression that this previous weekend was the last of the planned interdiction, but it turns out I am a week early.  Since things were kind of slow around the office, I decided to throw together some charts just for the fun of it:

First is an attempt to recreate the in-game market history.  Unfortunately, there are some volume spikes that make that unfriendly.  Also, trying to color in between the max/min lines is beyond my fu.  But it does create an interesting view of the market so far.  The funny thing being the price slopped up into the interdiction announcement, but quickly lost steam once the event went live.

Next up is a 'topes vs blocks chart.  Again, we're seeing that the announcement really pushed the price, but once the even drops, the activity is lack-luster.  The desired 1k/isotope price only sticks around for a short period, then rebounds right as the event starts, but quickly evaporates.

Thanks to JMP, I can build a pretty graph outlining the reality since before this whole ice SNAFU started.  The interesting thing here is how fuel costs are settling near pre-odyssey/post-announcement levels.  This represents a +10% increase from the summer normal.  Though the cost of operating a POS grows ever higher, the ramp in month-to-month costs is pretty flat and not really enough to cause issues in manufacturing.

Just thought I'd spit out some quick words/charts on the matter.  Hope to have a more complete analysis for TMDC in 2wks once the data has shaken out!

Cache Scraping SNAFU in Odyssey 1.1

This article was a pretty quick turn around for how many external resources I needed to reference:

• Out of Game Market Feeds Broken in Odyssey 1.1

First off, I'd like to thank the sources on this one.  Fuzzysteve was paramount in breaking the story and bringing the initial coverage.  Also, I have to thank him and K162space for fact checking me on some of the softer parts of the article in draft.

Also, I'd like to thank the other sources who were so good about getting back to me in a timely manner and sharing the kind of insights and links that really brought the piece together:

• Yann Ramin of EVE-Central
• pmchem of GARPA
• Tazuki Falorn of GARPA
• Entity of Reverence

Specifically Tazuki Falorn sending me the best response:

Not-an-official-GARPA-response: 

It's a ridiculous situation and one that should have been fixed years ago. Apart from CCP's general incompetence (see: anything to do with the current API server), there's no reason at all that this data shouldn't be available via some sort of API. Update it every x minutes with data from y minutes ago to limit the use in market bots. Cache data heavily. Spit out data in multiple formats, JSON would be nice since it's no longer 2005 but XML works if we really must. History data is even easier since it only changes once a goddamn day, ugh. 

Mood: mad because people bug me about goonmetrics not working and I'm lazy.

I didn't want to share that directly in the TMDC piece, but it's pretty on-target with most of the feedback I ever receive on this topic.  I didn't want to completely rehash my previous article, The Machines Are Taking Over, but ended up giving a TL;DR review of that piece to build this one

Furthermore, as I was pitching the article to Ali Aras, she was keenly interested in the topic as a CSM member.  Kind of highlighted how narrow I've been with my personal lobbying scope focusing almost exclusively on Mynnna.  We had a pretty decent discussion of the state of things as they are today and I expect to be in more regular contact with him as this topic continues to evolve.  I anticipate another pretty serious round of threats from CCP and counter lobbying from the community around the next expansion, especially since Stillman has hinted at changes to the EULA in the works since last spring.  I sure am glad the CSM is there to help promote some of the less-sexy, but generally important, topics.

Little Less Talk

If you haven't noticed, I have a terrible habit of dreaming up something, and never delivering.  I tend to get mired in the "do it right" steps that I run out of steam before I can actually solve any problems.

In an effort to break out of those bad habits, I set out this weekend to deliver a piece of code to solve my EMD_history problem.  A bit ago, I wrote a gdoc script that would pull data from EVE-Marketdata's history API.  This was crucial in delivering the graphs I've made for the entire summer, including my big TMDC moon article.

Unfortunately, this script is heavy and poorly suited for the gdoc environment.  It was prone to time-outs, and required a lot of massaging to get the data sets I needed.  Frankly, it was too weak for my needs.  And with CFC's Ice Interdiction scheduled to conclude this weekend, I need to be able to process the data quickly and effectively to get results posted while they are relevant.

Pricefetch sourcecode on github

Despite being more effort than originally expected, a usable script has emerged!  I don't have a witty name for it (it's way too buggy for full release), but I can at least now start scraping data from hosts and have it in a local version for getting from data to graph much faster than before.

Todo to make it more robust:

• Make a crash path.  Allow the program to resume where crashed
• More robust error catching: not handling some common exceptions (http requests)
• More complete arguments: currently only does The Forge, all items
• Add EVE-central dump crunching for stock charts and finer data
• Make a update path.  Check existing database for entries and only fill the missing ones

But today, I have a way to get that data into an intermediate faster and in larger quantities than before.  As soon as cache scraping is fixed, I intend to finish sending my alt to the busiest regions and pushing data to EMD for a more complete.

Also, once again, thanks to Fuzzysteve for helping me out with some SQL fu.  I added his help to the code snippet gallery