We spent several months on Natural Load Testing before launch, we also spent a considerable portion of the cash reserve WonderProxy had built up on smart people to help us along. We felt… pretty awesome. We started expanding our beta a bit further, and the feedback started rolling in. Key Feedback like: “It doesn’t work”, and “Something is broken” as well as silent feedback, where users got to a certain point in the process then stopped using the service entirely. That was… not so awesome.
The problem users ran into was quite understandable: A lack of ajax on our load testing page required users to manually refresh the page for results to appear. A quick glance at my wall calendar confirms that it’s no longer 2006, and this is an entirely reasonable position for our users to take.
I sat down, and started resolving the feedback as quickly as I could. My list of changes currently looks something like this:
- ****Ajax updates on Run and Calibration pages (“it doesn’t work”)**** DONE
- Better navigation, you have to go home after every step IN PROGRESS
- Delete things, Suites, etc. DONE
- *Graph results in real time*
- Estimated Time to Completion for test running page
- Configure threshold for reporting rows (currently hardcoded at 1000ms+) DONE
- Newer tests at the top of the run page DONE
- Plot tests to compare runs
- Fancy “load server” feature
- Deep Copy Suites DONE
- Change Suite Domain DONE
- Better identify the calibration runs on test suites DONE
- Check Time Elapsed column ms vs seconds DONE
- Name Runs DONE
- Add min width on response time
- allow editing test suites
- Calibration run can’t actually compare to anything DONE
- left align suite name (and make this a link) DONE
- make start time not a link DONE
- make editing run title only happen when you click the pencil
- zebra stripes for result tables! DONE
- Timings aren’t being saved in configure DONE
- Rename Tests (not just suites)
- Change domains on a Test level (not just test suite)
My original plan was to fix a few critical items that led to users thinking things were broken, then work on the beta invite list. As I finished each item, I looked at the list (which originally only contained a few items) and decided I needed to finish a few more before I could invite users.
It took me a few weeks to discover I was in an endless cycle. Users will always have feedback on how your site could be better. If I kept going like this, a year from now I’d have an incredibly polished site, with two users.
So, I broke the cycle. I just sent out another batch of invites, and I feel great.
but not that great about the typo in the From field of that email