We spent several months on Natural Load Testing before launch, we also spent a considerable portion of the cash reserve WonderProxy had built up on smart people to help us along. We felt… pretty awesome. We started expanding our beta a bit further, and the feedback started rolling in. Key Feedback like: “It doesn’t work”, and “Something is broken” as well as silent feedback, where users got to a certain point in the process then stopped using the service entirely. That was… not so awesome.

The problem users ran into was quite understandable: A lack of ajax on our load testing page required users to manually refresh the page for results to appear. A quick glance at my wall calendar confirms that it’s no longer 2006, and this is an entirely reasonable position for our users to take.

I sat down, and started resolving the feedback as quickly as I could. My list of changes currently looks something like this:

  • ****Ajax updates on Run and Calibration pages (“it doesn’t work”)**** DONE
  • Better navigation, you have to go home after every step IN PROGRESS
  • Delete things, Suites, etc. DONE
  • *Graph results in real time*
  • Estimated Time to Completion for test running page
  • Configure threshold for reporting rows (currently hardcoded at 1000ms+) DONE
  • Newer tests at the top of the run page DONE
  • Plot tests to compare runs
  • Fancy “load server” feature
  • Deep Copy Suites DONE
  • Change Suite Domain DONE
  • Better identify the calibration runs on test suites DONE
  • Check Time Elapsed column ms vs seconds DONE
  • Name Runs DONE
  • Add min width on response time
  • allow editing test suites
  • Calibration run can’t actually compare to anything DONE
  • left align suite name (and make this a link) DONE
  • make start time not a link DONE
  • make editing run title only happen when you click the pencil
  • zebra stripes for result tables! DONE
  • Timings aren’t being saved in configure DONE
  • Rename Tests (not just suites)
  • Change domains on a Test level (not just test suite)

My original plan was to fix a few critical items that led to users thinking things were broken, then work on the beta invite list. As I finished each item, I looked at the list (which originally only contained a few items) and decided I needed to finish a few more before I could invite users.

It took me a few weeks to discover I was in an endless cycle. Users will always have feedback on how your site could be better. If I kept going like this, a year from now I’d have an incredibly polished site, with two users.

So, I broke the cycle. I just sent out another batch of invites, and I feel great.
but not that great about the typo in the From field of that email :-)

Comments »

No Trackbacks
No comments

Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
Standard emoticons like :-) and ;-) are converted to images.

Hi, I’m Paul Reinheimer, a developer working on the web.

I co-founded WonderProxy which provides access to over 200 proxies around the world to enable testing of geoip sensitive applications. We've since expanded to offer more granular tooling through Where's it Up

My hobbies are cycling, photography, travel, and engaging Allison Moore in intelligent discourse. I frequently write about PHP and other related technologies.