Monday, December 31, 2012

Lessons in Accounting

Because I'm on a roll, I'd like to talk about some of the challenges I'm working through as my industrial nest egg keeps growing.

The Problem

I like growth.  I like doing bigger and better things, week over week.  This leads to me being heavily leveraged and with a very small ISK stockpile.   This leads to losing at least 1 day per cycle because of the need to sell nearly everything from the last week to pay for the next week.  I end up in a situation of limited mobility because of a lack of ISK stockpiles.  Also, once cash is sitting in the bank, I think I have carte blanche to make the next big step forward or leap on something juicy.  

For instance, I sunk nearly 3B into producing Mobile Large Warp Disruption Bubble IIs, which took nearly 3wks to pay off because of a continuing series of unforunate events.  That leaves 3B tied up and useless until the final product rolls off the line.  Also, with that big an investment, the cash out time will be slow too... all mistakes to learn from.  The upside?  Enough stockpiled BPCs to really pounce on that product if the need arises again.  Namely if my PVP characters go on a deployment to the other side of space.

Current Progress

I've been taking more diligent personal notes to track the week-by-week costs and projected profits.  I am less concerned with hitting the projected profit numbers as I am with charting and controlling the kit costs.  Also, this gives me a quick record of what I've done and a better handle on where I'm going.  Also, if I can keep better track of incoming profits, I can better manage other expenditures like BPO acquisition and PVP funds.

I would love to track this in gdoc, along with the rest of my tools, but I have no idea how to make snapshots that don't update with live data.  If I could just write product names and have the sheet take the NOW projections, I'd post it as a live list with a pretty chart.  I could just write the numbers by hand like a normal Joe.

The Future

The goal is to get with my more accounting minded partners and really wrangle in the inputs and outputs.  Currently my profit projections are a little too generous, and my costs aren't very exact either.  I'd like to be within +/- 5% on both.  I think +/- 1% on inputs is totally reasonable, but profit is going to remain a weak prediction until I can implement something more sophisticated.

Also, the goals should be to track individual pilot contributions by week and month.  As product cycles become less synchronized by week, it will be important to account events with regular tickmarks.  I anticipate ammo and ship production to slide to a 10d cycle as modules are pulled in to 6d.  These desynchronized cycles will require a larger operating budget in the bank.  But again, I am weak at accounting, and don't really want to have 2x the ISK required in stockpile, especially as the weekly budget grows to 10B.  I'm sure I will end up biting the bullet eventually, holding a cash stockpile over 60%, but I'd rather be smart about the issue rather than do the stupid-carebear thing.

 In the mid term, I'd like to be able to lock in a kit budget and plan kit cost growth from the stashed profits.  We'll see how well that works out.  Also, I'd like to chart discretionary expenditures and make sure the goals are on track.

As I figure out my way around charting, feel free to tune into progress here: https://docs.google.com/spreadsheet/ccc?key=0Atv4WV8DEJUPdE5OcnA4Z3JnekstNkZNWG1GZFVCUkE#gid=0

Year In Review

Because it's the cool thing to do this time of year, and everybody's doing it, I figured it was time to do the yearly retrospective.

Personally

This has been a year of both plunging both hands into industry, and nutting up for PVP.  It has been invigorating to be able to jump into the big times, PVP wise, with QCATS, and finally put a lot of that "noob friendly" dead weight behind me.  Not to say those that sponsor and raise noobs aren't awesome, but it's been a real joy to spread the wings of my ludicrous SP and really play with the big boys.  Unfortunately, the realities of PVP are kind of lame (2hrs of sitting around waiting for a fight, 15min fight).  When I can do enjoyable form ups, it's a blast.  I had a lot of fun putting my dreads to use, and providing logi in the big fights.  And the T1 cruiser changes have opened up the world of solo/very-small gang PVP, so I look forward to wasting some time there when work isn't destroying me.

Industry wise, it's been a huge learning experience.  As I come out of my shell and slowly become less of a fail coder, more and more tools open up to me.  From the joint tools I made with Aideron, to the development of Prosper here, I have learned a lot personally and look to really step up my game to the next level in the coming months.

Professionally

The overwhelming majority of my time this year was spent in the pursuit of heavy industry.  From my merger with Aideron Robotics at the end of 2011, to its growth and schism this summer, to heavy personal industry, it has been a year of heavy building.  Thanks to the professional relationships with TheAhmosis and Marcel Devereux from Aideron, I was really able to go from middling to prospering.

I did miss the year end goal of JF production by 3 weeks, but that's seriously not bad.  My comrades back in Aideron Robotics can attest to the 3 month build up and ultimate canceling of capital production, and being on track to still hit the goal in the same month is a massive improvement.

Lessons Learned

Tools Make the Project: After running a staff of 30+ heavy industrialists, the absolute truth learned there was that any project is bounded by the quality of tools.  Having Aideron's staffing tools was only the first step up, to really grow another set of shared tools was really required.  Also, the lacking environment of corp-level API tools means it's a whole uncharted DIY kind of territory.  This point is why Prosper got started.

Work Smarter, Not Harder: Somewhat related to the first point, not delegating responsibility properly or tracking real work became one of reasons Aideron fell apart.  With one or two directors losing entire days to managing logistics, you know you have a problem.  Also, with my personal emphasis on being in FW PVP, I needed to change my focus from maximum ISK to minimum clicks.  This is a major design consideration for Prosper, since I'd like to let humans play while computers work.

Death to PVE: This has been slowly coming for a long time.  As missions continue to be nerfed, and new content completely lacking, I've been actively avoiding ALL PVE.  Though FW was (and still kind of is) a cash cow, I just can't stomach all the boring ass PVE.  I'd really love to see some more PVE love, like incursions, but PVE content is expensive to produce and has a short shelf life.

Goals for 2013

Stop Reinventing the Wheel: There are a lot of awesome tools, and a lot of people with more freetime than sense have made a wealth of tools.  Sometimes there are niches, but team up when you can and collaborate!

Accounting is Important:  This is a personal weakness I am working through currently.  Working with Aideron Robotics and their toolset made me lazy.  Also, I have a tendency to desire constant growth, and this leads to being overlevereged.  Ahmosis has been a real great resource in keeping me on track in this regard.

Total Galactic Dominance: The goal for Prosper still stands, and it remains largely on track.  I have given myself a large buffer and decent roadmap, so the goal is to stay on track.  I continue to build the contacts list, and continue to grow the investment nestegg.


Saturday, December 29, 2012

EVE Tool Review: EVE-Skunk

EVE-Skunk is a repository of API keys, for the purpose of posting evemails publicly.  The main goal is to leak alliance emails for the lolz.  A friend on my G+ stream had a freakout when he found his own keys on the site.  And though in all real personal opinions, I think we should have a reasonable expectation of privacy, I also thing EVE gives us the sandbox to play out our psychopathic tenancies without doing real damage.

How It Works

Alliance Evemail is the least secure means of communication.  It can be accessed by any member's personal mail API.  The mail API is one of the more convoluted tools. So, somewhere between "I can't write my own tool" and "I'll just have an API with all the options enabled" comes the security vulnerability.  Then it only takes 1 mismanaged key and an entire organization's corp/alliance messages are made public.

EVE-Skunk takes APIs through an anonymous API entry.  So whether a spy has stolen those keys, or some user puts it in themselves, it's added to the tool's database and added to the wall.  Being a malicious site, there is no recourse to remove your data either.  

Bully Activism

It's no secret, EVE's API is very low priority among developers.  Also, it's a project that has been infamous for being passed from lead to lead, and an insistence on some very weird development strategies.  It's only through exploiting weaknesses and causing outrage that there is any hope to make it on CCP's sizable to-do list.  But even this outrage isn't enough to do the rework required to really secure the system... and with the current v2 API architecture, it will never be user-proof.

Marcel Devereux, developer of Aura for Android, has turned this into an artform.  Using the weight he caries with Aura to apply pressure in just the right ways.  When the API moved to HTTPS, but had a bargain SSL certificate, he pushed to fix it by notifying his users when they came looking to file a bug report.  When a function should exist or needs more features (FW tools for instance) he will add teaser panels to his app to get his users psyched for the function.  

It takes a nuanced approach to fix API issues.  CCP is understandably reluctant to drastically change any API, since this will mean considerable rework for the player devs.  It helps to know who are the right people to lean on, and how to create suggestions that are low-impact.  Sometimes big exploits must be done to showcase glaring holes, but without also contributing a realistic plan to fix it, CCP will never invest the effort to fix it.

Developer Trust

"It takes a lifetime to build a good reputation, but you can lose it in a minute."
-Will Rogers
Now, any developer can exploit the exceptional trust people put in using their tools.  It is easy to ferret away API keys to a central database and process them in complete secret.  It's easy to exploit extra data from people who cannot be bothered to understand what they are handing over... it's called Facebook.

If you want to build a widely used tool, community trust is absolutely critical.  There are enough code jockeys, enough paranoid users, that there is a significant force out there to keep you honest.  Unfortunately, there are also enough tinfoil-hat wearing weirdos that you will never be finished with the "how can I trust you?!" arguments.  The best things you can do are be open, be honest, and educate the masses... or tell them to take a hike if they don't like it!  You can't win them all.

How To Defend Yourself

Honestly, the auditing tools are pretty weak.  CCP gives an access log on your API page, but doesn't do much to figure out which key is to blame, or which queries might be malicious.  The first line of defense is use the customizable options.  You can have any number of custom keys, make one for each site.  Also, apply the same "don't fly what you aren't willing to lose" logic to your API.

In the end, it's just data.  V2 API's are read only and therefore the damage is only that someone knows your sekrets.  At the end of the day, it's all spaceship pixels and probably not worth losing your shit over.  Also, go get educated on what each of those APIs do.  CCP provides links to the API documentation on every option.  Also you can visit EVE-id.net if you want more information about how the APIs work.  Knowledge is power.

The Sandbox

As Stan over at Freebooted has put it, "EVE is the lair of psychopaths".  It provides us a sandbox to really explore all of human nature.  As I like to look at it, EVE is the only MMO where you can play a true villain  not just the backstory badguys.  It is important to separate the in-game personas from their real life players; pirates don't kick puppies, griefers are not terrorists.  It's just a game.

Personally, I love the metagame and what it inspires people to do.  EVE-Skunk is just another metagame creation... and that's why EVE makes it into the news.  Embrace the corporate espionage, step up your game, play in the sandbox.  

Thursday, December 27, 2012

Amazon Vs Mainstreet: The Null Industry Problem

Been fighting most the afternoon with @Mord_Fiddle on the #tweetfleet about nullsec industry.  You can read his blog over at Fiddler's Edge.

Though we disagree rather strongly on our particular positions on industry, it inspired me to bring up a new point in the ongoing "Farms and Fields" debate for nullsec.

The Goal

The goal is pretty simple: Bring industrial targets to the nullsec warzone. This would provide softer targets for enemies to raid, a reason for defenders to invest in their space (and its protection). A noble goal, and one I personally stand behind.

The Problem:

Supply is heavily Jita-centric.  People produce in high-sec, and final products are shipped to nullsec.  There is an organizational chasm between builders and killers.  The two halves work independently, and there has never been reason to challenge the arrangement.

This provides a few problems.  First is that nullsec operates without need for a defensible "home".  Second is the ever tightening spiral around a single trade-hub.  Both of these make the game bland and I agree both should change.

I could rehash what I've already covered, or the other proposals like "super-trit"... but instead let's look at a compelling analog.

The Analog:

The problem looks a lot like the IRL conflict between Amazon.com vs local Brick-and-Mortar.  EVE continues toward Amazon's offerings: a central one-stop-shop for everything you could ever want + easy shipping to the four corners of the galaxy.  This kills local economies and jobs, and the leverage Amazon has is incredibly hard to beat.

Now, before we start with the pitchforks and "NERF ALL THE THINGS", let's instead look at the parts and evaluate what they really mean.

Overnight Shipping

Though the current climate is centered around one distribution hub supplying everywhere does center on shipping, I don't think this is the crux of the issue, but a symptom of a larger problem.  

The current problem is that there is a painful inbalance between raw materials and finished products.  Either raw materials need to be compressed, shipped, and refined, or final products need to be shipped anyway to the waiting hands of troops.  Why bother with a heavy intermediate step when you're already shipping final product?  Disregarding actual cargo size or JF range, the compression rate is over 50x between products and their raw materials, you could eliminate JFs completely and not even touch this issue.

This is where the "super-trit" proposal gets its legs.  "If I didn't have to ship heavy trit [freighter can only fit ~75M trit per trip], I could build more in null".  I think the solution is simpler.  Though I'd like higher yield super ores for low-ends in null (+50% for Trit/pyre/mex) or an overall increase in low-end yield, I think reducing mineral volume by 10x will do better to solve the issue.  If the compression between raw materials and finished products is reduced, the balance shifts toward remote manufacturing.  This by itself does not solve the problem, but it does move the balance beam.

Best Prices Anywhere, Available 24hrs/day

Currently Jita is a gold standard for all products.  Also, the price differential between DIY and Jita market isn't terribly healthy.  T1 margins are razor thin to begin with, and the logistical hurtles to T2 manufacturing necessitate a heavy shipping step anyway.  For the trouble involved, you either are going to save very little or lose money by DIY.  

The DIY Value Add

My proposal here is nuanced, but humor me.  I think the root of this problem is throughput.  Namely, the time required for manufacturing T1 is near negligible.  

Today, a single manufacturer (given enough ISK) can produce far beyond the current global market volume weekly on the vast majority of products.  And this lack of bottleneck is the reason T1 manufacturing has no value.  If there is no "Bread and Butter" choice for production, and the only limiter is cash, there is no value to the time spent in manufacturing.

A quick run down of weekly single-character maximum T1 throughput vs current jita total volume:

  • 700 battleships per week (200 on market in Jita)
  • 2,000 frigates per week (1,000 on market)
  • 17,000 HAM launchers (1,600 on market)
  • 1,500,000 Scourge Heavy Missiles (2,000,000 on market)


Put the breaks on T1 production so that there is some intrinsic value in manufacturing.  Without scarcity, value cannot be added, and T1 provides no scarcity.  I do believe T1 should be faster than T2 (5:1 T1:T2 is my proposed value) but this focus on time being the scarce resource will push more people into private industry (POS) and provide a value difference between raw materials and final products.  By severely limiting throughputs, large volume or diverse orders will require teamwork.

Pair this with a fairer compression rate on minerals, and I think this would go a long way to changing the industrial atmosphere.  Adding complexity that requires teamwork, and gives opportunities to work within the current frameworks.  I'd also like to group these changes with a more mobile POS model, allowing for a "caravan" that doesn't need to be the purely stationary asset they are today... but my POS proposal will require its own post.

Risk V. Reward

This is a point I really have to lean on, and I think is overlooked in many industrial proposals.  Factories are expensive.  If they aren't expensive in equipment, they're expensive because of the work in progress (WIP).  As I said in my own Farms and Fields post, without castle walls to protect those infrastructure investments, the costs of shipping will always remain cheaper than DIY.  

It does boil down to "incentives".  People don't manufacture in null today because it's too easy to nuke the whole thing overnight, AND the costs of making the system work far outstrip the costs of shipping final product.  Even given a 50% reduction in time and materials, we haven't touched the core problems of security and logistics.  

I personally would love to see a means to disable production lines with raiding parties, but they would need to travel through a few layers of protections before getting to those factories.  And with titans and jump bridges making the ability to leap-frog so easy, and TZ warfare completely common place, I'm not confident in the current nullsec's ability to protect those assets.  Unless we can build castle walls, I don't see many new POS being brought to null for industry.

Some Notes on T2

Though I know CCP Fozzie is moving the conversation back to T1 hulls, there is still a significant need for T2 modules.  The material requirements for T2 are largely (50-80%) moon minerals, and their acquisition is designed to prevent any one entity from being able to effectively supply all those materials themselves.  This means there still needs to be a shipping step, and intermediate manufacturing.  

I don't think bringing "farms and fields" to null in the form of manufacturing will work without also bringing a contingent of T2 production with it.  And unless "peasants" are able to labor for those T2 components, there's no means for an efficient local source.  I like the top level concepts of moon mining, but with organizations like OTEC and the supply lines almost entirely passive, there needs to be a change.  My short term proposal is to move harvesters outside POS shields and make the process interruptable with subcaps.  But these steps only improve the moon mining mechanic, they don't contribute to industry directly.

I hope to have more T2 suggestions soon, but I fear it will need to be its own post.

Wednesday, December 26, 2012

Code Development: Moon Mining Tracker -- WIP

Sorry for the lack of posts.  I keep writing half an entry and then decide against publishing.  I should mirror Ripard's "Junk Drawer" feature at some point.  Also, IRL has been a real bitch, and the holidays have significantly cooled the fires here at work.  So I can slack of a little and write about what I've been working on.

Introduction

Through the end of November, my faction war corp QCATS and our allies in Drunk N Disorderly, had made it our goal to get into the moon mining business as a means for long term passive income.  I'm not particularly savvy on the exact politics, but we were snatching up assets in Syndicate and Aridia.  As the first phase of this project wound down, I got tagged for POS management because "you know how to spell POS".  Being the local POS expert, I was on the short list for managers.

Unfortunately, I know EXACTLY what that workload looks like and set out to do something to make it less hair-pullingly awful.  Also, it was a chance to really stretch my legs in Python, since I've been avoiding it like cleaning the garage.  

So I set out to build a relatively simple POS tracker that could be hosted on Google's AppEngine and provide a light weight webpage that people can check in on POS status at-a-glance.  The future hopes are to expand this to account-instanced functionality, so users can register and have their own private POS information.  But that's a much later stage, and I'm not particularly interested in marketing such a tool at this time.

Design

Data Structures - POS

The data structure is pretty simple at its face.  Provide a parent-child relationship where parents are towers and children are modules.  Then the individual children modules can have return values that tell us useful things about the setup.  Silo capacities, POS fuel, online status, all can be assigned in this class structure.

The EVE API

This was one of the easy parts (remarkably).  By first validating against the AccountInfo API, the tool can verify a correct API was loaded.  Once verified, the program can move on to each of the parts of setting up each tower's classes.  By querying the API in the program's MAIN section, then feeding the result webpage to Python's minidom, the DOM object can be passed into each crunch function rather than querying the API over and over.  

The flow looks like:
  1. Query APIKeyInfo for validity check
  2. Query StarbaseDetail for list of towers
    1. Process location information and tower status (online, offline, reinforced, unanchored)
  3. With list of Tower() objects, load specific tower data
    1. Tower type
    2. Fuel remaining
    3. Unique itemID's for towers
  4. Iterate over list of Tower()s and use AssetList to load children objects
    1. Module() load, type, and relevant information about each type
Once the data structure is loaded, some simple math can be run on the snapshot.  Since POS processes are an hourly occurrence, there isn't any need for live tracking.

Site Building

Since the whole tool is designed around a snapshot of how assets look now, the goal is to be able to refresh (or cache until it can be refreshed) the whole structure every time the site is refreshed.  It would be possible to cache the individual API calls and use them until they need to be refreshed, but again, without the need for up-to-the-minute data, that work is not justified at this time.

As for displaying data, the goal is a collapsible list of towers with their children silos and remaining fuel.  To render .jpgs for each module, with some intelligence as to which resources are depleting and which are filling.  Also, using the fixed rates of reactions, the goal should also be to display ETAs for full/empty for all progress bars.  I have not yet spent any time on generating the output site, so any stupid-proof graph builders people might know about, I'd appreciate the help.


In this example, the red bars are depleting stocks and the green are gaining.  Using the information from reactors, it should be easy to designate reactants and products rather than have to keep caches to watch which way things go.  Also, though I am a fan of the minimal, getting the right python modules to hook into the output should make it as pretty or plain as required.  Iterating over the intended inputs, procedurally generating the required website code, and presenting it in a pretty package as such, should be relatively simple.

Remaining Challenges

I want to have an admin panel that allows the user to name the various assets as they see fit.  Though my plan is to only allow control for tower names or module names, additional functionality can be added by expanding the asset classes.

Currently the plan is to only allow access to one client.  I'd very much like to add instancing for as many users as you'd like.  But there are some significant challenges to adding that function.  Security, public sharing keys... the ability to deal with other humans... it's a to-do.

"Parallel API" functionality.  I've tried to design in the ability to counteract the caching system by having multiple character APIs and have the program handle them seamlessly in the background.  There's a framework for it, but I am not wholly convinced it will work well and it's not important in this first release.

New Skillz

What's a dev-blog without covering "what we learned"?  This project is an adventure in demystifying the programming challenges.  Also, the motto going forward should probably be "just do it"... most of what I learned this time around was done by just putting code down and worrying less about the issues of how to get to the finish when I know almost nothing.

JSON

I've been a fan of XML for a while now.  It provides a human-readable framework to build fancy data structures.  XML is also a massive pain, with its tag structure, and the amount of rope it gives you to hang yourself is quite liberal.  JSON provides a much cleaner architecture to do very similar things.  Unfortunately the resources for learning JSON + Python are, to be blunt, shit.  

Some quick lessons:
  • Once a json object is parsed (json.load()), it can be referenced like the datastructure denoted
    • I used a "dictionary vector" setup: json[key1][key2][key3]=value
  • Iterating over a list with for-in returns the actual contents (object, value, structure), not an integer reference to its position in the list
  • No commenting in JSON.  Personal pet-peeve, but add "comment" key if you're anal

Object-Oriented Code

I missed the boat in programming class about Object-Oriented code.  There were some steps between A->B that did not click and I quickly lost taste for the practice.  Many examples and some hammering on my own time has quickly evaporated the haze around this style of code.  By providing classes for each component linking into eachother, and better understanding return types and how they can be passed around a program has been a godsend.  Also, getting away from the more hard-core elements of C, like pointer magic, has been a breath of fresh air.  Lastly, being able to generate classes with a whole range of attributes and tools makes modularizing the code much easier.

Keeping up with Progress

I hope to have most of the crunching functionality figured out this week (while work is quiet).  Maybe even get a rudamentary site up.  If you'd like to see what's going on, check out the repo:

I need to finish doing some clean up first.  Then the data-loading cruncher.  Then a basic website.  We'll see if it can't start looking like the intended product by the end of next week.