Monday, April 17, 2017

Aspiring Hari Seldon - Part 3 - Releasing REST Forecasts

Check out part 2

After playing with Prophet, I wanted a way to distribute the data more publicly.  I could have just incorporated it into our R templates and made it part of the show, but I felt like that was unfair to the general public.

(Though it required a ground-up rework) I used the Flask Restful app we were providing for EVE Mogul.  Though this turned out to be a complete rewrite, I learned a lot about Flask and testing to make things go.

What They Don't Tell You

One of the biggest pains of self-teaching is it's very easy to learn enough to be useful, but not enough to be good.  And the Flask documentation is particularly bad about giving up the barebones of what does what, but totally forgets to sell a viable project shape, leading to a lot of pain later.

Since the original OHLC feed was based loosely off a work project (as a way to test out corners on my own time), it repeated a design flaw.  Turns out there's a very particular Flask shape, and deviating from it causes a lot of problems.  

Some examples to avoid pitfalls:
By using the prescribed Flask structure, all the pieces from launcher to test make a lot more sense.  Particularly, testing was one of the more difficult pieces of the picture.

About The Endpoint

Because of the CPU-intensive operation of running predictions, I wanted to incorporate two features:
  • API keying
  • Caching
Though I'm happy to share the source, and open up to the community, I'm not ready for full-hog release.  TL;DR: There are a couple of Flask eccentricities, and gdoc integration could cause some serious issues.  Also, uncached performance can run north of 15s, which could cause issues for some platforms.

To help get around "walled garden" accusations, I've done a few things:
  1. I've left copious notes and automation on how to deploy the service on your own webhost
  2. I've provided API keys to some other market devs such as EVE Mogul and Adam4EVE
  3. I am happy to distribute keys on-request to other devs
The goal is to get the content out to the widest audience possible, even if the raw data is a little unwieldy.  And due to my limitations as a developer, this is my compromise.  

Predictions In The Wild

Adam4EVE



Though my API service is designed to give you soup-to-nuts everything you need to plot in the REST payload, Ethan02 over at Adam4EVE added his own DB to keep us honest.  I totally love this!  As of right now, it's still in their DEV branch, but expect to see more from them soon!

EVE-Mogul


Extending the existing OHLC candlestick plot, EVE Mogul will let you keep close tabs on what you're currently trading, and this is an excellent chance to gut-check your investments.

Conclusions

Check out the source: ProsperAPI

I didn't get to share all the other super-nerdy #devfleet stuff (like travis-ci integration, or PyPI release).  I will probably try to release more notes on python stuff directly on Medium going forward.

This was an eye opening project in a lot of ways.  This opens the door for more micro-service REST stuff in the future.  Also, I do plan to have the PLEX split covered before CCP releases it on May 9th.

Tuesday, April 11, 2017

My Space-Retirement

I hate reading this kind of blog, and I hate writing them even more.  One day I should show off how many drafts of this I have stuck in limbo.  My IRL trajectory has steadily diverged from EVE, and it's time to accept that and make plans known.

We'll save the how/why for a little later, but I want to address fans of the show first.  The Prosper show has been the thread keeping me tied into EVE, and our community around Prosper has continued to surprise me with their knowledge and enthusiasm.  The community we've been able to generate around this niche-of-a-niche has been extraordinary and I am constantly floored by all the players in the space-economic sphere.  This whole project has exceeded every expectation, and I have to thank every single audience member and technical contributor that's got us here.  None of this would be possible without a whole lot of help from wide range of people.

Before you unsubscribe and leave everything in the dust, do know that I still hope to keep active in the blogging/streaming spheres.  Though we are closing this chapter in EVE, we are looking upward to new topics that I think many will enjoy.  Though I can't commit to details yet, know that I'd very much like to turn this brand of analysis onto something more real, and help teach more in the vein of the #devfleet.

Expect the final EVE Prosper show to air in June.  I'd very much like to hit #100 before we close this chapter, but I don't want this to be the end of our little community.  I am still enthusiastic about the same things that started Prosper (economics, programming, datascience) and still want to bring a down-to-earth style to topics that might look insurmountably complex.  But I cannot continue to be that resource for EVE.

Thank you everyone that has made this show possible.  Thank you to Caleb/Rhivre at the SCC-Lounge, Jeronica at EVE-Mogul, and thank you to Dirk MacGirk and Materall at INN, for helping weave a community both in and out of game with us.  Thank you to the Capstable crew and the likes of NoisyGamer for helping us grow this srsbzns niche.  And thank you immensely to Randomboy/Kane, Chris Mellen, and Etienne Erquiline for all the behind-the-scenes technical work to build EVE Prosper into what it has become... and I am sure I'm still missing a cavalcade of others who have helped us stand and grow.


No one truly quits EVE, and I can't for certain say this is the absolute end... but after 12 years of play, and 5 years of content making, it's time to look to a new chapter.  Please, do join us as we bring this show to a close, and thank you to everyone who has made this possible.  There are a few remaining projects in the runway, so be sure to watch to the very end.

o7

Thursday, March 2, 2017

Aspiring Hari Seldon - Part 2

Been quite a while, but I have a follow up to this old post.

A Crash Course

Tinkering with prices is difficult, and most players may not understand why.  Though we all interact with the price of things, unspinning the mess of how and why becomes complicated fast.  

What's worse, the price of a thing doesn't follow traditional fitting tools, it's a collection of ups-and-downs day-to-day.  It's a solution to a complicated network of factors.  This is why I've had so much trouble designing forecasts; because starting point is critical, and the walking methods aren't strictly obvious.

After dancing around this problem for as long as I have, I've come up with a few best-practices for approaching Prosper's economic reporting:
  1. Make it normal or linear: hard math is hard, keep analysis as simple as possible
  2. Figure out connections: storytelling can quickly connect seemingly disparate points
  3. Assume players aren't dumb: everyone is trying to win
  4. Look for disproof: try to eliminate weaknesses and errors in tools

Prophet - A New Crack At Forecasting

My boss sent me a link to Prophet.  And, of course, I threw EVE data at it!
On the one hand, that PLEX prediction is pretty hilarious, while on the other, the injector forecast is very close to one I'd publish on the show.

So, what is going on here?  I still think we're running into the issue of putting linear-style modeling on a non-linear problem, but we're getting much closer to the gut-version I would like to illustrate on the show.  

What's more, we can use a previous lesson and use the forecaster to predict the day-to-day volatility as a second opinion.  Using a GBM-style method, we get this:
Getting a second opinion in this case is a good way to counteract the problem of runaway models; forecasts that get stuck in a runaway up or down swing.  Though it's not fool-proof, we're reaching a "good enough" level to actually start considering it in our tooling.

Let's Go!


As promising as these initial findings are, I still worry they aren't a good replacement for more robust methods.  I'm still investigating the following:
  1. Single-dimension: only using price data, not including volume for supply/demand accounting
  2. Limited test scope: only run on CREST history data, not extended history data yet
  3. Time series rigidity: designed for daily data
  4. No extended grading done: need to test predictions vs reality
But the future is promising.  There are some key features I'm loving in this library:
  • Built-in changepoint trending: predicting discontinuities is very powerful
  • Python/R library parity: easy development/testing
  • Built-in week/month/season accounting
  • EXTREMELY FAST
There's so much more to play with, and I'm excited to dive deeper.  I'm looking into including these forecasts sparingly into the Prosper show going forward, as a better way to illustrate my gut feelings.  And hopefully we can incorporate these forecasts in a more robust feature in the near future!
TypeNames redacted while debugging


Subscribe on Twitch and YouTube