Lessons in Accounting

Because I'm on a roll, I'd like to talk about some of the challenges I'm working through as my industrial nest egg keeps growing.

The Problem

I like growth.  I like doing bigger and better things, week over week.  This leads to me being heavily leveraged and with a very small ISK stockpile.   This leads to losing at least 1 day per cycle because of the need to sell nearly everything from the last week to pay for the next week.  I end up in a situation of limited mobility because of a lack of ISK stockpiles.  Also, once cash is sitting in the bank, I think I have carte blanche to make the next big step forward or leap on something juicy.  

For instance, I sunk nearly 3B into producing Mobile Large Warp Disruption Bubble IIs, which took nearly 3wks to pay off because of a continuing series of unforunate events.  That leaves 3B tied up and useless until the final product rolls off the line.  Also, with that big an investment, the cash out time will be slow too... all mistakes to learn from.  The upside?  Enough stockpiled BPCs to really pounce on that product if the need arises again.  Namely if my PVP characters go on a deployment to the other side of space.

Current Progress

I've been taking more diligent personal notes to track the week-by-week costs and projected profits.  I am less concerned with hitting the projected profit numbers as I am with charting and controlling the kit costs.  Also, this gives me a quick record of what I've done and a better handle on where I'm going.  Also, if I can keep better track of incoming profits, I can better manage other expenditures like BPO acquisition and PVP funds.

I would love to track this in gdoc, along with the rest of my tools, but I have no idea how to make snapshots that don't update with live data.  If I could just write product names and have the sheet take the NOW projections, I'd post it as a live list with a pretty chart.  I could just write the numbers by hand like a normal Joe.

The Future

The goal is to get with my more accounting minded partners and really wrangle in the inputs and outputs.  Currently my profit projections are a little too generous, and my costs aren't very exact either.  I'd like to be within +/- 5% on both.  I think +/- 1% on inputs is totally reasonable, but profit is going to remain a weak prediction until I can implement something more sophisticated.

Also, the goals should be to track individual pilot contributions by week and month.  As product cycles become less synchronized by week, it will be important to account events with regular tickmarks.  I anticipate ammo and ship production to slide to a 10d cycle as modules are pulled in to 6d.  These desynchronized cycles will require a larger operating budget in the bank.  But again, I am weak at accounting, and don't really want to have 2x the ISK required in stockpile, especially as the weekly budget grows to 10B.  I'm sure I will end up biting the bullet eventually, holding a cash stockpile over 60%, but I'd rather be smart about the issue rather than do the stupid-carebear thing.

 In the mid term, I'd like to be able to lock in a kit budget and plan kit cost growth from the stashed profits.  We'll see how well that works out.  Also, I'd like to chart discretionary expenditures and make sure the goals are on track.

As I figure out my way around charting, feel free to tune into progress here: https://docs.google.com/spreadsheet/ccc?key=0Atv4WV8DEJUPdE5OcnA4Z3JnekstNkZNWG1GZFVCUkE#gid=0

Year In Review

Because it's the cool thing to do this time of year, and everybody's doing it, I figured it was time to do the yearly retrospective.

Personally

This has been a year of both plunging both hands into industry, and nutting up for PVP.  It has been invigorating to be able to jump into the big times, PVP wise, with QCATS, and finally put a lot of that "noob friendly" dead weight behind me.  Not to say those that sponsor and raise noobs aren't awesome, but it's been a real joy to spread the wings of my ludicrous SP and really play with the big boys.  Unfortunately, the realities of PVP are kind of lame (2hrs of sitting around waiting for a fight, 15min fight).  When I can do enjoyable form ups, it's a blast.  I had a lot of fun putting my dreads to use, and providing logi in the big fights.  And the T1 cruiser changes have opened up the world of solo/very-small gang PVP, so I look forward to wasting some time there when work isn't destroying me.

Industry wise, it's been a huge learning experience.  As I come out of my shell and slowly become less of a fail coder, more and more tools open up to me.  From the joint tools I made with Aideron, to the development of Prosper here, I have learned a lot personally and look to really step up my game to the next level in the coming months.

Professionally

The overwhelming majority of my time this year was spent in the pursuit of heavy industry.  From my merger with Aideron Robotics at the end of 2011, to its growth and schism this summer, to heavy personal industry, it has been a year of heavy building.  Thanks to the professional relationships with TheAhmosis and Marcel Devereux from Aideron, I was really able to go from middling to prospering.

I did miss the year end goal of JF production by 3 weeks, but that's seriously not bad.  My comrades back in Aideron Robotics can attest to the 3 month build up and ultimate canceling of capital production, and being on track to still hit the goal in the same month is a massive improvement.

Lessons Learned

Tools Make the Project: After running a staff of 30+ heavy industrialists, the absolute truth learned there was that any project is bounded by the quality of tools.  Having Aideron's staffing tools was only the first step up, to really grow another set of shared tools was really required.  Also, the lacking environment of corp-level API tools means it's a whole uncharted DIY kind of territory.  This point is why Prosper got started.

Work Smarter, Not Harder: Somewhat related to the first point, not delegating responsibility properly or tracking real work became one of reasons Aideron fell apart.  With one or two directors losing entire days to managing logistics, you know you have a problem.  Also, with my personal emphasis on being in FW PVP, I needed to change my focus from maximum ISK to minimum clicks.  This is a major design consideration for Prosper, since I'd like to let humans play while computers work.

Death to PVE: This has been slowly coming for a long time.  As missions continue to be nerfed, and new content completely lacking, I've been actively avoiding ALL PVE.  Though FW was (and still kind of is) a cash cow, I just can't stomach all the boring ass PVE.  I'd really love to see some more PVE love, like incursions, but PVE content is expensive to produce and has a short shelf life.

Goals for 2013

Stop Reinventing the Wheel: There are a lot of awesome tools, and a lot of people with more freetime than sense have made a wealth of tools.  Sometimes there are niches, but team up when you can and collaborate!

Accounting is Important:  This is a personal weakness I am working through currently.  Working with Aideron Robotics and their toolset made me lazy.  Also, I have a tendency to desire constant growth, and this leads to being overlevereged.  Ahmosis has been a real great resource in keeping me on track in this regard.

Total Galactic Dominance: The goal for Prosper still stands, and it remains largely on track.  I have given myself a large buffer and decent roadmap, so the goal is to stay on track.  I continue to build the contacts list, and continue to grow the investment nestegg.

EVE Tool Review: EVE-Skunk

EVE-Skunk is a repository of API keys, for the purpose of posting evemails publicly.  The main goal is to leak alliance emails for the lolz.  A friend on my G+ stream had a freakout when he found his own keys on the site.  And though in all real personal opinions, I think we should have a reasonable expectation of privacy, I also thing EVE gives us the sandbox to play out our psychopathic tenancies without doing real damage.

How It Works

Alliance Evemail is the least secure means of communication.  It can be accessed by any member's personal mail API.  The mail API is one of the more convoluted tools. So, somewhere between "I can't write my own tool" and "I'll just have an API with all the options enabled" comes the security vulnerability.  Then it only takes 1 mismanaged key and an entire organization's corp/alliance messages are made public.

EVE-Skunk takes APIs through an anonymous API entry.  So whether a spy has stolen those keys, or some user puts it in themselves, it's added to the tool's database and added to the wall.  Being a malicious site, there is no recourse to remove your data either.  

Bully Activism

It's no secret, EVE's API is very low priority among developers.  Also, it's a project that has been infamous for being passed from lead to lead, and an insistence on some very weird development strategies.  It's only through exploiting weaknesses and causing outrage that there is any hope to make it on CCP's sizable to-do list.  But even this outrage isn't enough to do the rework required to really secure the system... and with the current v2 API architecture, it will never be user-proof.

Marcel Devereux, developer of Aura for Android, has turned this into an artform.  Using the weight he caries with Aura to apply pressure in just the right ways.  When the API moved to HTTPS, but had a bargain SSL certificate, he pushed to fix it by notifying his users when they came looking to file a bug report.  When a function should exist or needs more features (FW tools for instance) he will add teaser panels to his app to get his users psyched for the function.  

It takes a nuanced approach to fix API issues.  CCP is understandably reluctant to drastically change any API, since this will mean considerable rework for the player devs.  It helps to know who are the right people to lean on, and how to create suggestions that are low-impact.  Sometimes big exploits must be done to showcase glaring holes, but without also contributing a realistic plan to fix it, CCP will never invest the effort to fix it.

Developer Trust

"It takes a lifetime to build a good reputation, but you can lose it in a minute."
-Will Rogers

Now, any developer can exploit the exceptional trust people put in using their tools.  It is easy to ferret away API keys to a central database and process them in complete secret.  It's easy to exploit extra data from people who cannot be bothered to understand what they are handing over... it's called Facebook.

If you want to build a widely used tool, community trust is absolutely critical.  There are enough code jockeys, enough paranoid users, that there is a significant force out there to keep you honest.  Unfortunately, there are also enough tinfoil-hat wearing weirdos that you will never be finished with the "how can I trust you?!" arguments.  The best things you can do are be open, be honest, and educate the masses... or tell them to take a hike if they don't like it!  You can't win them all.

How To Defend Yourself

Honestly, the auditing tools are pretty weak.  CCP gives an access log on your API page, but doesn't do much to figure out which key is to blame, or which queries might be malicious.  The first line of defense is use the customizable options.  You can have any number of custom keys, make one for each site.  Also, apply the same "don't fly what you aren't willing to lose" logic to your API.

In the end, it's just data.  V2 API's are read only and therefore the damage is only that someone knows your sekrets.  At the end of the day, it's all spaceship pixels and probably not worth losing your shit over.  Also, go get educated on what each of those APIs do.  CCP provides links to the API documentation on every option.  Also you can visit EVE-id.net if you want more information about how the APIs work.  Knowledge is power.

The Sandbox

As Stan over at Freebooted has put it, "EVE is the lair of psychopaths".  It provides us a sandbox to really explore all of human nature.  As I like to look at it, EVE is the only MMO where you can play a true villain  not just the backstory badguys.  It is important to separate the in-game personas from their real life players; pirates don't kick puppies, griefers are not terrorists.  It's just a game.

Personally, I love the metagame and what it inspires people to do.  EVE-Skunk is just another metagame creation... and that's why EVE makes it into the news.  Embrace the corporate espionage, step up your game, play in the sandbox.  

Amazon Vs Mainstreet: The Null Industry Problem

Been fighting most the afternoon with @Mord_Fiddle on the #tweetfleet about nullsec industry.  You can read his blog over at Fiddler's Edge.

Though we disagree rather strongly on our particular positions on industry, it inspired me to bring up a new point in the ongoing "Farms and Fields" debate for nullsec.

The Goal

The goal is pretty simple: Bring industrial targets to the nullsec warzone. This would provide softer targets for enemies to raid, a reason for defenders to invest in their space (and its protection). A noble goal, and one I personally stand behind.

The Problem:

Supply is heavily Jita-centric.  People produce in high-sec, and final products are shipped to nullsec.  There is an organizational chasm between builders and killers.  The two halves work independently, and there has never been reason to challenge the arrangement.

This provides a few problems.  First is that nullsec operates without need for a defensible "home".  Second is the ever tightening spiral around a single trade-hub.  Both of these make the game bland and I agree both should change.

I could rehash what I've already covered, or the other proposals like "super-trit"... but instead let's look at a compelling analog.

The Analog:

The problem looks a lot like the IRL conflict between Amazon.com vs local Brick-and-Mortar.  EVE continues toward Amazon's offerings: a central one-stop-shop for everything you could ever want + easy shipping to the four corners of the galaxy.  This kills local economies and jobs, and the leverage Amazon has is incredibly hard to beat.

Now, before we start with the pitchforks and "NERF ALL THE THINGS", let's instead look at the parts and evaluate what they really mean.

Overnight Shipping

Though the current climate is centered around one distribution hub supplying everywhere does center on shipping, I don't think this is the crux of the issue, but a symptom of a larger problem.  

The current problem is that there is a painful inbalance between raw materials and finished products.  Either raw materials need to be compressed, shipped, and refined, or final products need to be shipped anyway to the waiting hands of troops.  Why bother with a heavy intermediate step when you're already shipping final product?  Disregarding actual cargo size or JF range, the compression rate is over 50x between products and their raw materials, you could eliminate JFs completely and not even touch this issue.

This is where the "super-trit" proposal gets its legs.  "If I didn't have to ship heavy trit [freighter can only fit ~75M trit per trip], I could build more in null".  I think the solution is simpler.  Though I'd like higher yield super ores for low-ends in null (+50% for Trit/pyre/mex) or an overall increase in low-end yield, I think reducing mineral volume by 10x will do better to solve the issue.  If the compression between raw materials and finished products is reduced, the balance shifts toward remote manufacturing.  This by itself does not solve the problem, but it does move the balance beam.

Best Prices Anywhere, Available 24hrs/day

Currently Jita is a gold standard for all products.  Also, the price differential between DIY and Jita market isn't terribly healthy.  T1 margins are razor thin to begin with, and the logistical hurtles to T2 manufacturing necessitate a heavy shipping step anyway.  For the trouble involved, you either are going to save very little or lose money by DIY.  

The DIY Value Add

My proposal here is nuanced, but humor me.  I think the root of this problem is throughput.  Namely, the time required for manufacturing T1 is near negligible.  

Today, a single manufacturer (given enough ISK) can produce far beyond the current global market volume weekly on the vast majority of products.  And this lack of bottleneck is the reason T1 manufacturing has no value.  If there is no "Bread and Butter" choice for production, and the only limiter is cash, there is no value to the time spent in manufacturing.

A quick run down of weekly single-character maximum T1 throughput vs current jita total volume:

• 700 battleships per week (200 on market in Jita)
• 2,000 frigates per week (1,000 on market)
• 17,000 HAM launchers (1,600 on market)
• 1,500,000 Scourge Heavy Missiles (2,000,000 on market)

Put the breaks on T1 production so that there is some intrinsic value in manufacturing.  Without scarcity, value cannot be added, and T1 provides no scarcity.  I do believe T1 should be faster than T2 (5:1 T1:T2 is my proposed value) but this focus on time being the scarce resource will push more people into private industry (POS) and provide a value difference between raw materials and final products.  By severely limiting throughputs, large volume or diverse orders will require teamwork.

Pair this with a fairer compression rate on minerals, and I think this would go a long way to changing the industrial atmosphere.  Adding complexity that requires teamwork, and gives opportunities to work within the current frameworks.  I'd also like to group these changes with a more mobile POS model, allowing for a "caravan" that doesn't need to be the purely stationary asset they are today... but my POS proposal will require its own post.

Risk V. Reward

This is a point I really have to lean on, and I think is overlooked in many industrial proposals.  Factories are expensive.  If they aren't expensive in equipment, they're expensive because of the work in progress (WIP).  As I said in my own Farms and Fields post, without castle walls to protect those infrastructure investments, the costs of shipping will always remain cheaper than DIY.  

It does boil down to "incentives".  People don't manufacture in null today because it's too easy to nuke the whole thing overnight, AND the costs of making the system work far outstrip the costs of shipping final product.  Even given a 50% reduction in time and materials, we haven't touched the core problems of security and logistics.  

I personally would love to see a means to disable production lines with raiding parties, but they would need to travel through a few layers of protections before getting to those factories.  And with titans and jump bridges making the ability to leap-frog so easy, and TZ warfare completely common place, I'm not confident in the current nullsec's ability to protect those assets.  Unless we can build castle walls, I don't see many new POS being brought to null for industry.

Some Notes on T2

Though I know CCP Fozzie is moving the conversation back to T1 hulls, there is still a significant need for T2 modules.  The material requirements for T2 are largely (50-80%) moon minerals, and their acquisition is designed to prevent any one entity from being able to effectively supply all those materials themselves.  This means there still needs to be a shipping step, and intermediate manufacturing.  

I don't think bringing "farms and fields" to null in the form of manufacturing will work without also bringing a contingent of T2 production with it.  And unless "peasants" are able to labor for those T2 components, there's no means for an efficient local source.  I like the top level concepts of moon mining, but with organizations like OTEC and the supply lines almost entirely passive, there needs to be a change.  My short term proposal is to move harvesters outside POS shields and make the process interruptable with subcaps.  But these steps only improve the moon mining mechanic, they don't contribute to industry directly.

I hope to have more T2 suggestions soon, but I fear it will need to be its own post.

Code Development: Moon Mining Tracker -- WIP

Sorry for the lack of posts.  I keep writing half an entry and then decide against publishing.  I should mirror Ripard's "Junk Drawer" feature at some point.  Also, IRL has been a real bitch, and the holidays have significantly cooled the fires here at work.  So I can slack of a little and write about what I've been working on.

Introduction

Through the end of November, my faction war corp QCATS and our allies in Drunk N Disorderly, had made it our goal to get into the moon mining business as a means for long term passive income.  I'm not particularly savvy on the exact politics, but we were snatching up assets in Syndicate and Aridia.  As the first phase of this project wound down, I got tagged for POS management because "you know how to spell POS".  Being the local POS expert, I was on the short list for managers.

Unfortunately, I know EXACTLY what that workload looks like and set out to do something to make it less hair-pullingly awful.  Also, it was a chance to really stretch my legs in Python, since I've been avoiding it like cleaning the garage.  

So I set out to build a relatively simple POS tracker that could be hosted on Google's AppEngine and provide a light weight webpage that people can check in on POS status at-a-glance.  The future hopes are to expand this to account-instanced functionality, so users can register and have their own private POS information.  But that's a much later stage, and I'm not particularly interested in marketing such a tool at this time.

Design

Data Structures - POS

The data structure is pretty simple at its face.  Provide a parent-child relationship where parents are towers and children are modules.  Then the individual children modules can have return values that tell us useful things about the setup.  Silo capacities, POS fuel, online status, all can be assigned in this class structure.

The EVE API

This was one of the easy parts (remarkably).  By first validating against the AccountInfo API, the tool can verify a correct API was loaded.  Once verified, the program can move on to each of the parts of setting up each tower's classes.  By querying the API in the program's MAIN section, then feeding the result webpage to Python's minidom, the DOM object can be passed into each crunch function rather than querying the API over and over.  

The flow looks like:

1. Query APIKeyInfo for validity check
2. Query StarbaseDetail for list of towers
1. Process location information and tower status (online, offline, reinforced, unanchored)
3. With list of Tower() objects, load specific tower data
1. Tower type
2. Fuel remaining
3. Unique itemID's for towers
4. Iterate over list of Tower()s and use AssetList to load children objects
1. Module() load, type, and relevant information about each type

Once the data structure is loaded, some simple math can be run on the snapshot.  Since POS processes are an hourly occurrence, there isn't any need for live tracking.

Site Building

Since the whole tool is designed around a snapshot of how assets look now, the goal is to be able to refresh (or cache until it can be refreshed) the whole structure every time the site is refreshed.  It would be possible to cache the individual API calls and use them until they need to be refreshed, but again, without the need for up-to-the-minute data, that work is not justified at this time.

As for displaying data, the goal is a collapsible list of towers with their children silos and remaining fuel.  To render .jpgs for each module, with some intelligence as to which resources are depleting and which are filling.  Also, using the fixed rates of reactions, the goal should also be to display ETAs for full/empty for all progress bars.  I have not yet spent any time on generating the output site, so any stupid-proof graph builders people might know about, I'd appreciate the help.

In this example, the red bars are depleting stocks and the green are gaining.  Using the information from reactors, it should be easy to designate reactants and products rather than have to keep caches to watch which way things go.  Also, though I am a fan of the minimal, getting the right python modules to hook into the output should make it as pretty or plain as required.  Iterating over the intended inputs, procedurally generating the required website code, and presenting it in a pretty package as such, should be relatively simple.

Remaining Challenges

I want to have an admin panel that allows the user to name the various assets as they see fit.  Though my plan is to only allow control for tower names or module names, additional functionality can be added by expanding the asset classes.

Currently the plan is to only allow access to one client.  I'd very much like to add instancing for as many users as you'd like.  But there are some significant challenges to adding that function.  Security, public sharing keys... the ability to deal with other humans... it's a to-do.

"Parallel API" functionality.  I've tried to design in the ability to counteract the caching system by having multiple character APIs and have the program handle them seamlessly in the background.  There's a framework for it, but I am not wholly convinced it will work well and it's not important in this first release.

New Skillz

What's a dev-blog without covering "what we learned"?  This project is an adventure in demystifying the programming challenges.  Also, the motto going forward should probably be "just do it"... most of what I learned this time around was done by just putting code down and worrying less about the issues of how to get to the finish when I know almost nothing.

JSON

I've been a fan of XML for a while now.  It provides a human-readable framework to build fancy data structures.  XML is also a massive pain, with its tag structure, and the amount of rope it gives you to hang yourself is quite liberal.  JSON provides a much cleaner architecture to do very similar things.  Unfortunately the resources for learning JSON + Python are, to be blunt, shit.  

Some quick lessons:

• Once a json object is parsed (json.load()), it can be referenced like the datastructure denoted
• I used a "dictionary vector" setup: json[key1][key2][key3]=value
• Iterating over a list with for-in returns the actual contents (object, value, structure), not an integer reference to its position in the list
• No commenting in JSON.  Personal pet-peeve, but add "comment" key if you're anal

Object-Oriented Code

I missed the boat in programming class about Object-Oriented code.  There were some steps between A->B that did not click and I quickly lost taste for the practice.  Many examples and some hammering on my own time has quickly evaporated the haze around this style of code.  By providing classes for each component linking into eachother, and better understanding return types and how they can be passed around a program has been a godsend.  Also, getting away from the more hard-core elements of C, like pointer magic, has been a breath of fresh air.  Lastly, being able to generate classes with a whole range of attributes and tools makes modularizing the code much easier.

Keeping up with Progress

I hope to have most of the crunching functionality figured out this week (while work is quiet).  Maybe even get a rudamentary site up.  If you'd like to see what's going on, check out the repo:

https://github.com/lockefox/EVE-Prosper/tree/master/POS_Track

I need to finish doing some clean up first.  Then the data-loading cruncher.  Then a basic website.  We'll see if it can't start looking like the intended product by the end of next week.

Farms and Fields: Bringing Industry to Nullsec

One of the common whines I hear in industry circles is "Industry is nigh impossible in null sec".  Between lack of slots in sov-space (player owned and controlled) and the impossibility of logistics to and from POS, heavy industry is basically absent from nullsec.

Without industrial investment, there are no soft-targets to hit.  Without "farms and fields", there is nothing for smaller gangs to "burn".  And though I agree with the sentiment of enriching gameplay through more targets, I fear for the viability of industry across EVE.

I fear the changes because the first trumpet of 0.0 space is "risk v reward".  That everything should be inherently better only on the measure of risk.  That call is always paired with "nerf high sec", which would destroy (I think) a valuable balance between builders and destroyers.

The Way It Is

Currently, the modus operandi is buy in Jita, and ship to the warzone.  I don't think this is a terrible way to go, but secondary hubs are drying up and Jita is becoming more and more monolithic, which I think IS unhealthy.  

There is also a significant issue of investment when it comes to dropping a factory.  On the surface, there could be a cost/benefit analysis between dropping POS in LS/0.0 without the need for standings.  Unfortunately, there is no added benefit, so the entry cost of high security (standings) is just so much better than providing a target you will eventually lose.  This incentivizes high-sec heavy-industry corps.

Instead the benefits are in shipping.  It would currently make much more sense to sponsor a near-by high-sec industry corp that could ship to a null hub than try and support the infrastructure inside null space where it can be interrupted.  I personally think this is the place where improvements can be enacted  making it easier to make a factory mobile, allowing for a rear-operating base, rather than "farms and fields".  But if the CSM is going to push for "farms and fields" then we should look at what that would mean.

Supply Chain

This is the obvious bottleneck in sov-space industry.  Even if slot/POS issues were solved by tweaking some DB numbers, the supply chain problem would remain.  Currently, to haul the materials needed for production, compression is required.  Compound that with the trouble of reprocessing and then delivering to a POS.  This all adds up to "frack it, I'll buy it in Jita and haul the finished product".  Some proposed solutions:

Super-Veld

Currently, mining rates on all ores are nearly flat on a per-hour rate.  Also, thanks to the hellfire toward bots, the per-hour rates are pretty lucrative for miners.  Since rates are flat, why risk a mining fleet in PVP space when you can harvest vanilla ores in safe space?  If you're going to need to ship compressed materials anyway, why not just skip the mining op all together and reduce the steps?

Since the hauling volume bottleneck is in low-end minerals (tritanium, pyerite, mexallon), the first suggestion in solving the supply chain bottleneck is to make super-yield low-end rocks that can take advantage of tools such as rorquals.  This would allow for harvesting materials in much higher yields locally, incentivizing manufacturing locally to then monetize out those hauls.  This would remove a leg of the supply chain into null.

Personally, I think this is a very good idea, since I personally believe the low-end prices are way too high at the moment, and it is causing the price of things to be distorted.  There would need to be some finesse as to how people could invest in their space to get this super-veld, but I think there are a lot of opportunities in this department to ease up several bottlenecks.

Refinery Issues

Both in WH and null sec, there's a big problem of "Water water everywhere, but not a drop to drink".  The issue lies in refinery availability.  Any way you go, there will be a yield hit, and if you're stuck hauling large volumes around from op->outpost then still hauling heavy loads from outpost->POS, then we haven't eliminated a supply chain leg.  

I think the issue can be solved with a heavy revamp to mobile refineries.  Increase their holds to 100k-500k, increase the yield to 75-90% flat (no skill improvement).  Even make the refinery types restricted to ores and make cycle time proportional to load.  By giving a mobile refinery where ore can be refined near the factory, a supply chain is significantly improved.  This also puts fully functional factories in the crosshairs of enemies, allowing for raiding targets.

Castle Walls

The problem with industry is it is diametrically opposed to PVP.  Without significant security, it's far too easy to burn down a lot of hard work.  It's like saying we should build a tank factory in Afghanistan because we need tanks there... when instead we can put that factory somewhere secure and ship in the tanks.  

I agree we should incentivize thriving headquarters for alliances, but the volumes alliances require are not worth putting on the line.  Unless we can build fortresses, there is nowhere to put factories.  Unless valuable BPO's can be secured, no one will risk the factory.

POSibilities

With current POS mechanics, the choice is between factory and security.  If you want to have a tower that has any defensibility, you can only half-load it with industrial capabilities.  I don't think this mechanic is directly bad, but without significant fuel savings the cost/slot and logistical hurtles of providing enough manufacturing throughput (as well as technical organization to make several towers work together).  Furthermore, the hassle of management would require returns that are significantly better than "just buy it" to really justify the risk.

As for proposals, I'd start with a PG->CPU conversion module, nullsec only.  This would allow a tower to be hardened and expanded to allow a secure base of operations.  This could be abused to make dickstars even more dickstar-y, but I think a balance could be reached to incentivize the intended behavior (higher stacking penalties on shield hardeners?).  I would also like to see a method to disable/slow production through enemy gang activity, but that's probably asking too much in the game mechanics department.

Risk vs REWARD

The main point still stands: why build in null when I can ship instead?  Without incredibly significant bonuses to build speed, material requirements, and ore yields, why would anyone take the risk.  Also, with the requirements for industry being so diametrically opposed to PVP, why would any organization tolerate the shift in paradigms.  Already PVP organizations have near-zero tollerances for missing CTAs due to "carebear excuses", and without sticking to a production schedule, I don't see how any organization can hope to have enough materials to justify the investment.

There is a significant oil-and-water world collision when it comes to PVP and industry.  The types of people who will put up with the industry grind are not the type of people who can be counted on for frontline PVP.  A lot of orgs will not find the losses acceptable when they appear on enemy kill mails.  Efficiencies will drop as carebear activities are attacked.  Space honour and image will be challenged as blocs require carebear corps for support.  

Also, my proposed changes still don't address the T2 supply chain, which would still be unsustainable in null.  The moon supply chain is designed to avoid monopolies.  Also, invention would require hauling of either invented bpcs or large groups of datacores into null, since there are no supplies coming from that space.

It's because of these two meta issues, image and real-return, that bringing significant industry to null is not a trivial matter to fix.  Without very significant overhauls to industry mechanics (most not for the better), there will be no change in the status quo.  Unless balances can be struck between killers and builders, where both can exist in the same space, then there will be no progress away from Jita. 

The Joys and Pains of Stockpiling

A topic I abhor, but a common point of contention among heavy industrialists.  It has always been a major point of contention among my teammates about what is the best way to stockpile.  From the utilitarian to the economic, there is a range of reasons to stockpile.  Unfortunately, it's something that can quickly get out of hand and cause issues both expected and unexpected.

A disclaimer: I am an engineer, not an accountant.  I feel confident speaking on the top-level economics of the topic, but am not reliable on the hard-core finance tools (heges, futures, derivatives, etc).  I am not publishing this as a guide, but instead discussing my opinions on applying the topics to your own EVE experiences... Use with caution, regard with a grain of salt.

Real Life Analogs

Stockpiling, or hedging, allows for insuring against future price spikes.  Airline fuel is the common example: if I expect prices to be higher in the fall, I will hedge against that price and find a way to lock in a lower price in advance of that spike.  

Since EVE lacks the nuances of modern financial products, hedging is done by stockpiling.  Whether that is hoarding products for price spikes, or just keeping a backlog of raw materials to quickly react to price sways in manufactured product.  

In EVE, I don't dare, for one second, to claim I know how to read the commodity markets.  In the traditional sense of hedges to "insulate against price spikes" I am a miserable failure.  Instead, let's look at the different ways we can stockpile to make the industrial process less painful.

Before Stockpiling

The cost of stockpiling is ISK.  ISK that can't be used for other processes, ISK that isn't being turned into more ISK, ISK that can be stolen/lost/depreciate.  It's very easy to have a project that has more value in stockpiled goods than actual net products.  This is the reason I treat stockpiling so gingerly.

Since the goals should be to simplify the industrial process (hauling, building), and try to make more ISK, pick your stockpile carefully.  Having large stores of commodities like Tritanium might make heavy production more agile.  Holding back product for the swings of market ups-and-downs might mean more ISK/unit.  Having a store of built T2 components might better utilize build times.  Just remember, that's ISK not in the wallet.  Also, the further refined the product is, the harder it is to cash-out the stockpile if something goes wrong.

Raw Material Stockpiles

Personally, I am a big advocate of stockpiling low-end minerals, and high-volume PI materials.  I should hold on to more moon components and T1, but I find those products are either light enough to transport easily or have such a small price differential, they aren't worth holding on to.

A practice I am trying to get better at is becoming smarter at stockpiling things that are essentially free (blue print copies), or take very little liquid ISK to hold on to (datacores/decryptors).  Stockpiling these intermediates serves to make the T2 manufacturing process much more agile.  Even T2 BPCs are pretty cheap to hold on to.  Also, they may be a theft risk because of : shiny: but would be such a pain to monetize, I can't imagine someone having the patience required.

Product Stockpiles

Since I am so bad at extrapolating trends, and tend to try and keep all my ISK working at all times, I don't like stockpiling finished product.  But, for the supply-oriented manufacturer, holding the same margin as the expected profit is like holding free goods.  This can be useful to portion off 10-30% of products for going straight into corp use.  

My problem with corp-use product stockpiles was two fold.  On the one hand, managing getting product into the hands of members is a pain.  On the other hand, keeping a diverse enough stockpile is a massive headache.  Why do we have one gun but not another?  Why can't I have the whole ship comped?  Trying to balance both an ISK-generating manufacturing house AND a direct PVP supplier is a lot of headache.

Cash Stockpiles

Cash is king.  Unlike the other stockpiles, cash can be turned directly into product instantly.  Whether that's ship equipment for the corp, or materials for manufacturing.  Though some will argue it's not the most efficient to just sell-order 100% of your materials, or buy-order 100% of your products, there's nothing like having piles of cash to just get it done.

Personally, this is the means I like to stockpile.  Unfortunately, as I mentioned earlier, I have a personal bad habit to invest stockpiles too readily and end up over leveraged.  Also, cash is really easy to move and steal. So be careful how you hold cash.

Security and Stockpiles

As countless corp thefts have highlighted, a stockpile is a target just like any PVP engagement.  Any pile of goods can be stolen by anyone.  Some thoughts to keep in mind:

• Difficulty of monetization: Good stockpiles are hard turn into cash
• Enormous piles of trit are heavy to transport
• T2 BPCs take a lot of time and/or material to manufacture
• Intermediaries might not have a player market (T1, RAM)
• Security of materials: Corp tools are your friends
• Secure hangers by need of access
• Utilize POS lab use/take rules to protect materials
• Alt stashing cash
• Remember: nothing is 100% secure once you involve human interaction
• Utility of stockpile
• Don't be afraid to reduce stocks if inputs > utilization
• No reason stockpiles can't be fluid.  Change to suit needs

Engineer Engineer

Pardon me a moment of off-topic.  I promise, development blogging will resume soon(tm).  I have a couple entries still in draft as I finish them.  Hopefully I can finish them this weekend.

I would like to highlight a trend I find disturbing in the technical jobs sector.  The haphazard slapping of "engineer" on any title to make the job sound more impressive.  Being a formally trained, and practicing electrical engineer, from a family of engineers, I take the title seriously.  Unfortunately, since STEM education has fallen out of favor until very recently, I don't believe most of the general public has the vaguest idea what an engineer actually does.

Hint: It has nothing to do with driving trains.

About The Author

I currently work as a Test Engineer in Probe (Wafer Sort) at a memory fabrication house.  My current project is developing the first ever production-volume phase change memory (PCM) part.  Furthermore, my job is tied to part reliability as we beef up the process for higher volume.  This means I spend my day designing tests and experiments to properly stress the part, and deliver the data for fab to improve the process.

Though in practice, my job is not that impressive, I maintain my credentials.  I have a BSEE from Colorado State University, and have passed the Fundamentals of Engineering exam as the first step toward earning my PE.  I hold membership in IEEE and several of their societies, namely Networking (ComSoc), and Power & Energy Society.  

The Trend

I have seen many job postings, and have friends in many technical positions, where the term "engineer" has been attached.  From programmers to technicians, the trend seems to be an inflation in title and/or a deflation in responsibilities.  

Floor technicians called "solutions engineers".  Over-the-phone tech support called "Technical Support Engineers".  Website designers called "Network Engineers".  

This trend is infuriating when trying to get a handle of what companies actually do.  When your company touts significant technological or industrial advancements: ground-breaking new tech, innovation, revolutionary new service... I then expect when I see open "engineering" positions, that the company requires the kind of broad-base, profit-minded, data-oriented skill set an engineer brings... Or at least be willing to pay the premium that comes with that skill set.

If someone called themselves "doctor" without credentials, they would be ridiculed.  If someone represented themselves as a "lawyer" without accreditation, they could ruin their client's life and be held liable.  If a "plumber" or "electrician" misrepresents themselves, they could cause havoc... yet "engineers" can be anyone from a car mechanic to sales person?  Seems a little broken.

What Does "Engineer" Mean?

Engineering has become synonymous with technical expertise, but the title is so much broader than that.  Engineering comes with a host of other responsibilities and liabilities.

Design

Engineers design the systems, tools, and infrastructure we use every day: every system and part in a car, the bridges and highways we drive on, the city planning for sewage and flooding, the internet this blog is posted on.  Every day you touch thousands, of systems that engineers designed.  These systems are considered critical for the lifestyle we enjoy, our livelihoods, or even our lives.

Analytics

No system gets better without data and analytics.  Though most breakthroughs come from the laboratory and scientists, real world optimizations are made by engineers.  Timing traffic lights to decease pollutants, designing more efficient computer networks to serve data faster.  Without a feedback path of design->analytics->optimization, no system would be robust enough to survive the ever changing challenges of the real world.  Engineers provide the expertise to gather, analyze, and apply this data to continue improvements and innovations.

Ethics and Liability

This is most important to me, when defining the title "engineer".  When a bridge collapses, everyone from the welder to the designer will be investigated.  If there is a fault in the design, and an engineer signed the drawing, he can be held liable, even face jail time.  And beyond life-and-death consequences, engineers have incredible power to cost companies outrageous sums.

A dark joke in engineering circles:

What's the difference between a doctor and an engineer?
When a doctor screws up, only one person dies

Business Sense

It is the duty of engineers to take the experimental from R&D and turn it into products for the public.  Scientists can study in schools and labs all day, but until a plan is developed to monetize the technology, it does very few people any good.  It is the duty of engineers to analyze the elements of production and bring the theoretical into reality.  Also, it is the duty of engineers to remind their clients that nothing is free, and technically every feature can be bought... but the price can be unfeasible.  

My favorite professor, Dr George Collins liked to note

Scientists research and develop new technologies, and businessmen make money.  But engineers bridge the space between to turn ideas and technology into sellable products.

Who Should be an Engineer?

I am the first to state that formal education should not be the only path to success, and engineering is similar.  Many fields and positions might have elements of engineering.  Many people might work their way off the production floor and into design.  Instead, I believe that if the position embodies the spectrum of work outlined above, then they are an engineer.  

A certain element of education cannot be overlooked.  Without the broad-base interdisciplinary education that every classically trained engineer receives, it is hard to supplement the body of knowledge that engineers are called to have.  But the motivation to go find out and learn should not be seen as monopolized by education or institution.

I don't mind calling programmers engineers, or inventors engineers.  Many embody the hallmarks above.  But  without the liability or risk to really break things, it's hard to justify the title "engineer".  Without impact, without risk, the title engineer is only embelishment.

The Price Tag of Engineering

All of this expertise comes at a price.  Engineering enjoys one of the last bastions of "old style" business; where employers are actively invested in gathering specific individuals and doing whatever it takes to get them and keep them.  This investment in human capital does mean there is a real life price tag on engineering positions.  Not only in salary, but also in incentives and talent acquisition.  

Any position's pay should reflect the value added.  Engineers, being an all-in-one package with both a host of skills and the commitment to bare the load of responsibility, should be adding significant value.  In my company, several times a year individuals in my division save millions of dollars per quarter on their process improvements.  Money that would have been spent otherwise on inefficiencies.

I want to call upon companies to analyze their titles and really think about whether that "engineer" is really deserved.  Under this scrutiny, I am not sure if I could keep my engineer title for the real work I do, but this is still a legitimate question.  If you wouldn't ship a candidate in on the company's dime, if they will not add several-times-over the value spent on them, then I am not sure if the position really needs an engineer.  

TL;DR

The part I really want to drive home again is the ETHICAL responsibility that comes along with engineering. The price of failure is what measures engineers.  Landing a $2B lander on Mars.  Keeping power on at a data center automatically.  Designing and maintaining bridges and subways.  These are the topics that cost serious cash, puts lives on the line, and require more than average-Joe to design, maintain, and improve.  

Sorry for the rant, we will return to your regularly scheduled spaceships talk soon(tm).  

Everything You Never Wanted to Know: Google Spreadsheets

Spreadsheets and Spaceships; the classic rebuttal to EVE online's title.  This guide is a pre-post for another upcoming series reviewing existing tools.  This is to pair with the writeup for EVE-Central.  These two articles were written in parallel, so there will be a decent amount of overlap.

Most of the lessons covered in this guide are Google-specific commands.  Some will work in excel, but several functions will be Google-only.  We will once again be skipping the laser-focused specifics and remedial topics, focusing instead on those that may have a spreadsheet (or dozen) and wants to improve their existing tools.  Also, we will cover a few best-practices for design so as not to get bogged down with the slew of new features.  There will be a set of example spreadsheets to illustrate concepts.

Resources are at the end of the article.  After the break to save some screen space.

Introduction

Google Drive provides a suite of easily-shared common office tools.  Documents, presentations, forms, spreadsheets, everything a budding small business would need to thrive, and none of the cost of MS Office.  Also, being cloud-hosted comes with some additional benefits.  The cloud allows for incredibly easy access of live web data, without having to write clunky excel scripts.  Also, being able to share these documents means that you and your corp can control access to tools.  This guide is purely for the Spreadsheets toolset.

Basic Functions 

I write this because there was a suite of basic functions I did not know about when I switched into Google Spreadsheets.  Feel free to skip ahead if these get filed in the "duh" category.

Anchor reference ($)

This delimiter allows you to use the drag-copy function while anchoring specific references so they will not drag.  For instance, building 4 condors.  By making the first reference (mineral*$qty), you can then drag-copy across the other minerals (rows) to complete the reference in 1/8th the time.

Technically, the $ freezes whatever its next to:

• $A1: Will freeze COL to A
• Drag-copy COLs: reference will remain A1
• Drag-copy ROWs: ROW will update, but COL will remain A
• A$1: will freeze ROW to 1
• Drag-copy COLs: COL will update, but ROW will remain 1
• Drag-copy ROWs: reference will remain A1

&, JOIN(), REGEXEXTRACT()

Being code minded, I often want to programatically update string cells. Manually filling every cell with their real names, and then having to do it over again for the T2 variant sucks. Also, different groups might need to be added/stripped in more elaborate methods, hence what these string manipulators are for. Using these tools, you can add a lot of quick functionality for only a small amount of work.

• &: allows you to join strings (and references) together.  Think java string printing "hello "+x+" world"
• Great for joining small things together 
• A1=Target Painter I, A2=A1&"I"-->Target Painter II
• Great for adding cell references into string commands
• "api.eveonline.com/api/CharacterList?vcode="&vcode&"&apikey="&apikey
• Join(): works like most scripting join().  Delimiter, string_array
• Convert to csv string
• Join item id's for importXML()
• Use like an iterated &
• RegExExtract()
• Allows for regular expression parsing
• Regular Expression cheat sheet

Array Functions

• Transpose: swap rows/columns.  
• Example: make a list a header
• Sumproduct(): Multiplies each element by corresponding element then sums the outcome
• Example: cost of materials

ImportXML()

ImportXML is the single most useful function for EVE tools.  ImportXML allows you to automatically load nearly-live EVE market data.  Using the numerical typeID for items you want to track (like minerals, products, ships, etc), you can query the APIs of sites like EVE-central.  Also, you can filter out specific data using an xpath query.

Some resources for the uninitiated:

• Sample Spreadsheet
• typeID lookup
• EVE-Central API resource

Some notes to keep in mind:

• Tends to update on-the-hour
• Can force update by changing URL (add/remove "http://" from address)
• DB is most likely to fail request on-the-hour due to cron traffic
• Cannot script cron (as far as I know)
• Join() is your friend
• Join maxes out at 100 entries
• Google's complexity limit is 50 importXML calls.  For very-large sheets, be smart about queries
• Use join (or combine join's)
• Query more data per request:
• Ill-advised xpath: "//sell/min"
• Better xpath: "//sell" or no xpath
• More data per call means less calls for the same data
• xpath "|" does not work like you think it does.  Will concatenate on the END of the first query, not beside the first query #WorkingAsIntended
• Each importXML call will make the spreadsheet slower
• See best-practices notes for how to cope with these limitations

WARNING: Do not try to queryXML() EVE API data.  Whoever set up those XML's was a turd and did not make them xpath compatible.  It can be done, but you need to write an app-script to parse the data, existing functions will not do it.

QUERY()

Query is a tougher tool to use, but incredibly powerful for simplifying a lot of the manual work of updating spreadsheets.  Query allows a nearly-complete SQL-like interface for parsing data.  This becomes EXTREMELY useful for some more eloquent features or further staving off the need for custom code.  I am by no means a SQL pro, but a few simple cases can add a lot of power to your spreadsheet.

Uses:

• Quickly reference price data with a simple command
• Query('Price data'!A:X, "SELECT B where A='"&cell reference&"'",False)
• Group useful data:
• Automatically grab highest and lowest data without needed complex IF()'s
• List collections of data on a sortable variable
• Sort all the frigates out of a list of ships
• Group suitable products between a certain cost

Notes:

• When referencing strings (item names), make sure to include single quotes around value
• ='string' and close SQL query with a double quote
• Not fully formed, only need SELECT and sort criteria, no FROM
• Finicky just like SQL, some frustration will ensue
• Be careful about overrunning boundaries   CONTINUE() will stomp over cells and give precedence to whatever it is continuing

Best Practices

-Or- How I learned to stop worrying about complexity and trust importRange()

Import functionality takes a lot of processing power and will slow down your sheet significantly.  Pile on Query() and a lot of sumproduct() calls, and your sheet will not process.  Instead, make your more complicated tools modular and use importRange() to recycle common functionality.  

Personally, I use importRange() on my personal priceDB.  Also, importRange() is great for bringing in any other data you might care about.  Also, it allows you to protect the sensitive sheets (APIs, work you don't want to share) and import the useful data.  Protection isn't foolproof, it's a security-through-obscurity solution.  Technically, only authors with access to the source material can import into their controlled sheets... but this doesn't give feedback to the source owner as to who is remotely accessing the data.

The biggest lesson I can share is modularity.  Have one sheet for prices, one sheet for t1 building, and so on. Hosting ALL THE THINGS on one sheet will only make a crawling behemoth.  Also, have some patience with importRange().  If the range is dependent on something slower, it may take a minute or two to get the right values and propagate them through the sheet.

Being smart about sharing

Google gives 3 levels of access control, additionally with read/write access for each.  Though the obvious level is to switch the entire sheet on or off, there are some middle ranges that might be useful.

• Hidden sheets
• As long as sheets are protected, they cannot be viewed when hidden
• Collaborators still have access to hidden sheets
• Make a second public sheet
• Hide 100% of the secret sauce in another sheet
• Make a public shell that only shows final data

Personally, I wish "public" could force a sign on anyway, but only invited collaborators will show as non-anonymous.  

A note about Drive-Mobile

Because I am a non-stop nerd, I just want to share that Drive-Mobile is a great place to VIEW these complex sheets... but a terrible place to EDIT them.  Edit with EXTREME caution.  Functions cannot be written in mobile, and some dependent references may not populate as intended.

Resources

• Google Doc function help
• SQL for dummies
• XML for dummies
• Example Spreadsheet

A Wise Man

One of my terrible habits is a constant use of quirky sayings.  Today's entry is dedicated to the saying:

"A smart man learns from his own mistakes.  A wise man learns from the mistakes of others"

Today's blog is dedicated to the developer of Aura, the leader of Aideron Robotics, Marcel Devereux.

About Marcel

Marcel is an extraordinarily gifted contributor to the EVE development community.  Aura is the defacto Android app for all things EVE.  His personal corp site is one that all organizations should strive for. A suite of powerful tools that enable an enormous amount of feedback to his corp members and leadership.

After working with Marcel, I found that he was extremely charismatic and bull headed.  This is not to deride his behavior, but strong men are imposing and strong wills often clash.  I partnered with him at the start of 2012 to push my industry operation from middling to larger scale, and on that front we succeeded greatly.  We managed to generate the program I had always dreamed of, reaching sales figures upwards of 50B/mo.  

Also, in this time, Marcel taught me an enormous amount to bring my own game into the big leagues.  With his extensive knowledge of Google's wide assortment of tools, I was able to upgrade my capabilities and jump into brand new possibilities.

Also, I took away a series of lessons:

1. Don't Develop Alone

This first point is the most difficult in any creative project.  The project is your baby, no one can nurture it like you.  This may be the way to start, but it becomes a massive burden.  The more people that rely on your product, the more they demand of you.  If you're not careful, all of your time is sunk into a side project, and there's no time left to explode spaceships.

This was a major point of friction when working with Marcel.  I was not skilled enough to help take some of the load, and my demands were complicated and hard to implement.  This was one of the major reasons our project closed down; too much demand on one person.

2. Keep It Simple, Keep it Clean

I think it's extraordinarily important to go peruse Aideron Robotics website.  It is the foundation that the entire corp is built around.  The major reason it's so successful?  It's exactly what you need, and not overbearing.  The forums are light-weight (and mobile friendly), the application tools are easy to read.

Design has been a critical component to Aura and Aideron Robotics website.  It is a component that I have trouble replicating.  Many of my own tools get close, but end up failing to wall-of-text.  Time will need to be spent in the foundation focused on the front-face of Prosper.  Also, being an electrical engineer, my code tends to be utilitarian, not efficient... so focus will need to be paid to avoiding code pitfalls.

3. Be Honest, Be Open

One of the temptations that arose from being tied to Aura was to add restrictions to protect Aideron.  Farming APIs, barring war targets from the app, unscrupulous data farming for profit.  Marcel made a point to directly state that ANY use of Aura for covert personal gain would mean the complete death of the app and its place as a primary resource for players.  He is completely open about the behavior and operation of the app, while still retaining control over the development.

4. Be accessible

The reason Aideron Robotics was my favorite corp to be in was connectivity.  The forums were active, we chatted both in-game and out constantly, it was very easy to chat business or pleasure with anyone in the corp.  Also, with the extensive Google integration, there was a connectivity that fostered further connectivity thanks to the various tools that all interconnect between G+, Google docs, and IM.

5. It's a hobby, not a job

The first rule of any EVE venture should be for the pursuit of fun.  As soon as the GAME stops being fun, there is no reason to continue.  It's easy to get caught up in commitments to others, to think "they are depending on me", but there is no reason to be in this kind of project if it's not fun.  

Everything You Never Wanted To Know: Invention

Since it will be nearly a month until I have any decent results to show off, I might as well fill the blog with useful articles.  I hope to make this a weekly (or twice per week) segment on topics the general EVE public may not know about.  The sort of "mysterious black box" that scares most of the PVP elite off because of :numbers:.  These won't be in any particular order, so as to not take 3 weeks to get to the interesting parts.

Introduction

I could start with T2 BPOs and their weird niche, but it's been 4 years since invention replaced the BPO lottery.  If you've been around EVE that long, you've heard all there is to know about them.  Instead, we will focus on the mechanics around invention and the sort of industry that builds up around it.

At its most basic, invention is a chance-based mechanic that turns T1 BPCs into T2 BPCs.  These T2 BPCs are negative efficiency, so they take more materials and more time than BPOs, but make up for this shortfall with volume.  On the surface, the system looks like: throwing away money on datacores THEN throwing away money on inefficient building.  On the contrary, BECAUSE of that inefficiency, there are enormous profits to be had.

I cannot stress enough how much invention is a VOLUME game.  The temptation for small-time industrialists and T1 is to mine what you need and build it yourself... making it all free (wrong, by the way).  Invention has the dual problem of not being feasible to DIY materials (moon materials, enough datacores, etc), and produces very large quantities of product, that no one person is going to churn through in any short while.  Seriously, what are you going to do with 100 Ballistic Control Unit II's?

:Math:

Inventing

There are some excellent guides on the exact :math: on sites like Grismar and Chruker.  Go play with their tools to figure out the exact numbers.  I will review the top-level :math: and boil down the concepts.

One of the basic tenants of probability is: do something enough times, the probability falls out to a percentage.  Flip 1M fair coins, you'll be pretty damn close to 50/50.  Don't focus on the streaks, failures, successes; only focus on the long-term picture.  The secret to normalizing datacore costs is to invent enough to be able to use the P[success] numbers without tracking real successes.

This means settling on a build quantity BEFORE you start inventing.  With all-4 skills (except ships), this is essentially 50% success rate (48.5% if you want to be a stickler).  Factor in the datacores as a build expense by using:

• (cost per attempt) * (1/P[success]) = 1 Blueprint
• 1 T2 BPC (usually) = 10 runs
• Invention cost = [(datacores per attempt) * (1/P[success])]/(BPC runs)

Also, some answeres to basic FAQ's for invention mechanics:

• Copying: always max-runs (except ships).  T2 BPC output is modified by (runs/max runs)*(T2 BPC max runs)
• (30 runs/ 300 max) * (10 runs T2) = 1 run T2 BPC
• ME/PE research?  
• NO: T2 BPC output is only governed by decryptor or lack thereof.
• 0/0 BPO's are highly encouraged for invention BPCs
• Data Interfaces, Datacores, Decryptors, Meta Items
• Data interfaces: not consumed.  Essentially a "key" to allow invention
• Datacore: consumed every invention attempt
• Decryptors: consumed (1 per attempt), OPTIONAL
• Changes ME/PE and resulting runs
• Meta Items, OPTIONAL
• T1 has zero effect
• meta 1-4 increases P[success]
• 1 consumed per attempt

Manufacturing

The real bottleneck in T2 invention is manufacturing.  Due to the -4 PE penalty, and the requirement to consume the entire BPC to make up the datacore cost, build times can range from 8hrs/10-run to 3d/10-run or even more.  Therefore, a focus must be paid on how much throughput one builder can actually make.  Consider the following:

• 1 BPO can yield 20 copies at a time
• 20 copies * ~50% success rate = 10 T2 BPCs
• Build time of ~1d = 5-7x per week that "kit" can be run (with adv mass production 4)

The obnoxious thing that crops up is that no T2 BPCs match a normal human schedule.  2, 4, 8, 16, 24 hours fit well into a normal schedule.  Many products fit in 18, 22, 28, 36 hour schedules (even with POS bonuses).  This can make throughput tricky to calculate, since most people won't log in at 4a, or from work, to make sure their production chains are running 100% efficiently.  Take careful consideration of build time to understand what is actually possible to fit into a schedule.

Also, do not forget that T2 requires intermediate products.  Components like microprocessors are MUCH cheaper to build yourself, but take manufacturing time away from T2.  This is why T2 is such a great team-effort since work can be combined and more T2 can be produced as intermediates are handled in more efficient batches.

Kits

The easiest way to account for all the costs is to treat the process like a more-complicated T1 build.  Instead of only requiring minerals, your T2 kit will need to look like:

• BPCs (POS makes these essentially free)
• Datacores
• Components (or the moon materials to build them)
• R.A.M.
• T1 counterpart
• Minerals

This way, at the end you have purchased the above kit for X and sold the product for Y.  Profit = Y-X.  

Advanced Topics

Ships

Ships are a special case.  The P[success] is much lower, and the invention time is much longer.  This makes the previous assumption of "high volume simplifies probability" less valid.  Also, the problem of manufacture being the bottleneck is switched with invention, since the time required to generate BPCs is much longer than the product build time.  Also, with volumes reduced to 1 ship per blueprint, the existing BPO stock has a higher impact on profitability (though still a small one).  Lastly, ships are sexy, everyone wants to build ships, many people are not accounting their math right.  Very fine attention must be paid to profitability.  There is a lot of ISK to make on T2 ships, but it requires a lot more effort.

Tricks to keep in mind:

• Only make single-run BPCs.  Ships default to 1-run BPCs on success, max-run concept does not apply
• Get a POS: invent time is halved, copy time bonus helps too
• Component build is very large: Almost half the build time for ships is components
• Expensive kit costs (normalized by 20-copies per BPO):
• Frigs: 200M/BPO
• Cruisers: 1B/BPO
• BS/BC: 2.5B/BPO
• SOME ships benefit from decryptors, but MOST are barely worth the extra trouble

Decryptors

Decryptors modify the efficiency, P[success], and runs per T2 BPC.  Accounting for them is the same as datacores ((1 decryptor + datacores)* (1/P[success_modified]))/(runs_modified).  But since the resultant BPC is changed in ME/PE, it's not as simple to just add that cost to the existing calculation for the default -4/-4.  The math for handling the calculations is a little obnoxious, but EVE HQ's Prism tool can handle negative ME/PE correctly, so that's a good base point to start with.

Some tricks:

• The 3rd tier decryptors are most often the most useful
• Increases efficiencies to -1/-1
• Increases P[success] to ~60%
• Does not modify runs
• Basically ONLY for ship invention (reduces datacore costs)
• Will increase kit cost, but will also increase profit/unit

Conclusions (TL;DR)

The main point to take away from this guide is T2 invention is a large volume game.  Because the invention mechanic forces large batches to account for datacores, you will be producing items in the hundreds.  This is one of the reasons people shy away from invention; they cannot DIY all their equipment because of the various restrictions.

As such, I have always run invention programs as "cash generators".  Even if we went and bought the items we sold back, getting the cash to get the small volumes required for ship fitting was so much easier than accounting for surplus and stockpiling.  Also, to get effective stockpiles you would need to run a very broad program building nearly everything, which takes a ton of man-power and cash.  I will cover more topics on accounting in future segments

Previous Tools

As I work to generate the very first draft kind of tools, I'm filling this blog with various higher-level topics.  Once I can start showing off the innards, expect the topics to become more technical.

Today, I'll walk through the iterations leading up to this point.  Mostly the gdoc spreadsheets, since I don't want to host the original excel sheets.  Each of the headings below is a link to the project.

Initial Industry sheet

This is the very-latest revision of this tool.  The entire thing is extremely kludgey.  Originally the thought process centered around a few different problems/ideas:

1. "Group" products: many blueprints of a type share datacore types.  No need to train ALL the sci skills, just focus on grouped types
2. Can't make a shopping list in a spreadsheet.  There are so many extra types of materials, there is no quick way to boil down inputs
3. Dependent Products: Need moon materials to build components, need T1 to build T2

The end result is this first draft tool.  Constantly needing to paste one more thing is why it's completely unreadable under the hood.  Furthermore, the main interface was extremely confusing and took forever to explain to people.

Even worse was for nearly 2 years, this tool was not automated.  So, despite providing a mostly-useful shopping list, not all kits were used every week.  This lead to manually adding the values I actually needed (since I didn't want to delete the "mixes" I had made up).  Also, all price data had to be added by hand.  This gave no insight to actual profitability, only cost.  Only after I joined Aideron Robotics did I add automation through eve-central.  Price automation made the tool tolerable, but we quickly outgrew its effectiveness.

Build All the Things Sheet

This is the culmination of a team effort between TheAhmosis and myself to get a way to look over the ENTIRE T2 market in a few quick clicks.  Beyond just price automation, we devised new metrics to fine tune the product selection process.  

• Throughput: What one inventor can build in 6days.  This is rounded in such a way to estimate human-capabilities
• 16hrs to build: no one will wake up early or log in from work to flip.  Round up to 24 hours
• 26hrs to build: same problem as 16 hours, better to round up to 48hrs
• %Throughput: throughput/volume.  Margin means nothing if you can't sell what you build.
• %profit: Isk efficiency.
• Revenue: Throughput * Profit.  What is pocketed for the work
• Slot Cost: up front costs to build
• ISK/hr and adjusted ISK/hr: ISK/hr is just regular build time, adjusted accounts for throughput rounding

Also, locking down the under-the-hood calculations allows me to share this sheet with anyone and protect the work I've done.  This way I can save others from building similar tools, and allow a central place for number checking.  I still use this sheet to pick my products.

The downside is there is a LOT of data working in the background, and it's not optimized for efficiency.  This results in the price estimates falling over on bad XML calls, working slowly from sheet-to-sheet, and sometimes timing out.  Thankfully, the complexity seems to be within Google's bounds, so despite being slow, it's 90% effective.

EVE_Market-crunch Perl Project

This was an exercise in XML tied to some IRL work I was having trouble with.  This has a lot of power, but being Perl, is pretty hard to iterate.  This was the initial framework I tried to build for the goals of Prosper.  A quick run down of functionality:

• Priceparse.pl: processes market data to build an XML list of base material prices
• Marketcrunch.pl: uses the manufacture.xml list to build price data (like priceparse)
• Kitbuilder.pl: uses manufacture.xml and producers.xml to make a general kit list
• Manufacture.xml: all the T2 material requirements for everything (except rigs)
• Component.xml: All the materials for components (cap and normal) + RAM
• t1.xml: all the mineral requirements for t1 building (assumes researched)
• kits.xml: output of kits.  Contains per-char material list as well as a total shopping cart

This suffers the same fate as many of my first attempt code.  It's very kludgey and half-formed.  I ended up having to abandon development due to wanting to help Marcel on his updates.  Both projects fell apart, and I'm stuck with a mediocre tool set here in Perl.  I still use the kit builder to manage my week-to-week building.  Unfortunately, maintaining the build requirements in XML is incredibly time consuming and not exactly stable or sustainable.

FW LP Calculator 2.0

This was another large undertaking that suited itself decently to a spreadsheet.  There are a few interesting features going on here:

• Pseudo-DB of rewards: the All Rewards sheet is exactly that.  I can then use QUERY() on those values to do some interesting features that would not be possible otherwise
• Shopping list: Currently broken.  This allows people to pick their rewards and quantities to get info on both expected income and tag requirements

Unfortunately, this is pushing right against the complexity limits allowed by google.  As such, it was only possible to add Minmatar rewards to this sheet.  The lesson taken away from this project is making complicated spreadsheets modular.  Using the importRange() function, each background tool can be isolated and compartmentalized.  Also, that means I can have a central ALL THE PRICES database and import small parts of what I actually need.

I did start a newer version with all the rewards and a lot more efficiency, but with the faction warfare changes to LP coming in December, it is no longer a priority.  Also, the hope is to add LP functionality to the Prosper project once industry tools are finished.

API Crunching App-scripts

After talking with Cameron Zero at length about RVB's fitted ship program, he shared their sheet with me.  Though he was automating price data, asset stockpiles were still being entered by hand.  I had already had a half desire to have EVE API data in gdoc, so I jumped on trying to work something up for him

• CharList(keyID, Vcode): Returns character name, charID, corp name, corpID for a given API key
• stationAssets(keyID,vCode,charID,stationID): Returns all assets for one character at one station (Default Jita 4-4)

This is only the start of what I would like to provide for the EVE community in Google Spreadsheets.  But with CREST on the horizon (and a lack of simple ways to handle id->name translation inside app script), this project is on hold.

Day 0: An introduction

Welcome to the blog for Prosper, a suite of web tools for EVE online.  Or, at least, that is the goal.

About the Project

EVE Online boasts one of the deepest and broadest market and crafting systems of any MMO currently available on the market.  As such, there are many different levels people can participate in the creation, trading, moving, and destruction, of all the goods inside the game.  

There is a very active community of private and public tools, but sometimes you just have to do it yourself.  After wrestling with spreadsheets of increasing complexity for years, and trying to make due with half-formed solutions that were only okay, it's time to build something for exactly what I need.

Though the first revisions will be purely selfish (as I figure out what I am actually doing), the overall hope of the project is to publish the tools for others to save them the trouble of trudging the same path.  Though, with anything in EVE, everything has its price.

The hope is I can build a foundation in the form of an internal database and a lot of the EVE API handling and get assistance down the line to do the "pretty" parts.  For now, this serves as a personal learning project and may open up further as goals change.

About the Author

I am an electrical engineer, living in Boise ID.  The very first point I'd like to make is I am barely competent when it comes to code.  The purpose of this blog is to both log my thought process and allow a path to work back what I did right and wrong.  

In game, I am Lockefox.  Having been in EVE online since late 2004, I've done a wide range of activities, but settled into industry as an obsession   I've worked through all sorts of levels of activity, starting with the basics of any HS carebear, moving through high-volume production, and currently settling into industry as a means to fund PVP.  But knowing my cyclical nature for EVE projects, heavy industry will knock again, and I want the tools to be able to execute the full scope of my vision the next time I jump back in.

Goals for the Project

1. Generate and maintain a database of detailed price data.

Though tools like eve-central and evemarketeer provide great information for trading, they are severely lacking on the production front.  There is no way to query historical price data, no tracking "cost" vs "value".  Furthermore, other tools might track buy/sell data, but nothing accurately tracks build costs.  Also, without any build-oriented tool, there are no reliable breakdowns to drill into real costs to find the real culprits of price spikes and drops.  A dedicated database allows me to both get EXACTLY what I want, and clean up some of the useless stuff.

2. Allow in-depth tracking of products as they are built

While working with the team at Aideron Robotics, we had a powerful corp tracking tool that tracked member contributions.  Taking the ideas applied there, I hope to expand tracking to include a whole host of capabilities:

• Individual revenue tracking: what does each inventor cost/produce
• Hours tracking: Who is working on what, and allow for wage calculations
• POS utilization
• Allow tracking in account groups: allow alts across corps to track in one "project"
• Detailed buy/sell tracking: Not only for precise accounting, but trader tracking for individual productivity tracking
• Promote a decentralized architecture: Allow the project to transcend the boundries of an individual corp. 

3. CREST Automation

This will be gated on CCP's willingness to open up the appropriate CREST access to allow automation.  The pinnacle goal is a totally automated machine that picks products to build, buys inputs, arranges assets in kits, manages contracts for shipping/delivery.  The machines should reduce player input to "install and deliver jobs" as far as industry goes.  This will reduce the need to pay people for untrackable work, and reduce a lot of the tiresome bullshit that kills large-scale production

4. Total Market Dominance!

The end purpose of these tools is to allow for extremely high-volume production of equipment for all of EVE.  Though the first goal is to use Jita to turn work into cash, the longer term goal is to have independent teams supplying all of EVE.  This project should help take some of the bullshit and confusion out of industry and allow anyone (or their alts) to contribute to a central production behemoth.  Also, additional predictive tools to track the rise and fall of various products should be part of this goal.

5. A Front-End that is expandable to allow private accounts

Though running a massive carebear corp is its own brand of "fun", the tools should be accessible to others who still demand to do it themselves.  This should allow an automated billing system that requires users to contribute (ISK, most likely) for the processing time.  Also, an end goal of a mobile app (at least mobile-friendly website) should be considered, but will be a long ways off.

Why "Prosper"?

When it comes to titles, I am terribly uncreative.  I wanted something EVE enough to point to something RPish, but all the low-hanging fruit was already taken.  Most of the every day EVE lingo is taken by apps (Aura, Neocom, iClone).  As ambitious as this project is, I might as well pin my hopes to a Jove corp.  Also, the name highlights the end goal.  

I'm not focus-grouping this shit... just making it up