We spent several months on Natural Load Testing before launch, we also spent a considerable portion of the cash reserve WonderProxy had built up on smart people to help us along. We felt… pretty awesome. We started expanding our beta a bit further, and the feedback started rolling in. Key Feedback like: “It doesn’t work”, and “Something is broken” as well as silent feedback, where users got to a certain point in the process then stopped using the service entirely. That was… not so awesome.

The problem users ran into was quite understandable: A lack of ajax on our load testing page required users to manually refresh the page for results to appear. A quick glance at my wall calendar confirms that it’s no longer 2006, and this is an entirely reasonable position for our users to take.

I sat down, and started resolving the feedback as quickly as I could. My list of changes currently looks something like this:

• ****Ajax updates on Run and Calibration pages (“it doesn’t work”)**** DONE
• Better navigation, you have to go home after every step IN PROGRESS
• Delete things, Suites, etc. DONE
• *Graph results in real time*
• Estimated Time to Completion for test running page
• Configure threshold for reporting rows (currently hardcoded at 1000ms+) DONE
• Newer tests at the top of the run page DONE
• Plot tests to compare runs
• Fancy “load server” feature
• Deep Copy Suites DONE
• Change Suite Domain DONE
• Better identify the calibration runs on test suites DONE
• Check Time Elapsed column ms vs seconds DONE
• Name Runs DONE
• Add min width on response time
• allow editing test suites
• Calibration run can’t actually compare to anything DONE
• left align suite name (and make this a link) DONE
• make start time not a link DONE
• make editing run title only happen when you click the pencil
• zebra stripes for result tables! DONE
• Timings aren’t being saved in configure DONE
• Rename Tests (not just suites)
• Change domains on a Test level (not just test suite)

My original plan was to fix a few critical items that led to users thinking things were broken, then work on the beta invite list. As I finished each item, I looked at the list (which originally only contained a few items) and decided I needed to finish a few more before I could invite users.

It took me a few weeks to discover I was in an endless cycle. Users will always have feedback on how your site could be better. If I kept going like this, a year from now I’d have an incredibly polished site, with two users.

So, I broke the cycle. I just sent out another batch of invites, and I feel great.
but not that great about the typo in the From field of that email