Engineering Judgement Framework > Execution > Defining Success
ENGINEERING JUDGEMENT FRAMEWORK
LEVEL 1
|
EXECUTION
Defining Success
Success cannot be achieved if it was never clearly defined.
Once you’ve framed the problem, the next question is:
“How will we know if we solved it?“
Engineering maturity is not just about building correctly. It is about defining success correctly. Clear success criteria act as a compass — guiding decisions, shaping priorities, and preventing drift. Without this clarity, teams can deliver impressive work and still miss the mark.
Prefer a video instead of reading? Here it is.
TABLE OF CONTENTS
What Does Success Look Like
Define success in terms of real-world impact, not completed tasks. Shipping a feature is an output. Improving user experience, reducing errors, increasing reliability, or saving cost is an outcome.
When success is framed as outcomes:
- Decisions become clearer
- Priorities become easier to defend
- Work aligns with value, not activity
Example
A team might say their goal is to launch a new search feature. That’s an output, and hence can’t be the measure of success.
A better definition of success would be:
- Reduce the time users take to find what they need by 30%
- Increase successful searches by 50%
- Lower support tickets caused by search failures
This way, the success will be measured by whether the feature actually improves user experience.
Without having clarity on the outcome, teams risk building something that is good, but not effective.
Success Criteria Must Be Measurable
Defining success is not enough. It must be measurable. Vague goals create vague outcomes.
“Improve performance.”
“Enhance user experience.”
“Make the system more scalable.”
These sound meaningful, but are directionless without numbers.
- What does improvement mean?
- By how much?
- By when?
- For whom?
A clearer definition of success would be:
- At least 95% of users experience a response in under 200 ms
- Maintain this performance for up to 20,000 concurrent users
- Keep failures below 0.1%
If success cannot be measured, it cannot be evaluated. And if it cannot be evaluated, it cannot guide decisions.
Example
A payments team was tasked with “improving checkout performance.” For weeks, they made incremental changes: query optimizations, minor caching tweaks, refactoring parts of the API. Each change felt productive, but there was no clear way to tell if they were actually succeeding.
Finally, the team worked with the client and defined an explicit success criteria:
- 99% of checkout requests must complete in under 300 ms
- The system must handle 5,000 concurrent checkout attempts during peak sale events
- Payment failure rate due to system errors must remain below 0.05%
- Performance must remain stable under load tests simulating 2x projected traffic growth
This clarity changed everything.
First, it exposed architectural gaps. Their synchronous payment verification flow could not reliably meet the 300 ms target under peak concurrency. Instead of continuing small optimizations, they redesigned the flow using asynchronous confirmation and queue-backed processing for non-critical steps. The architecture shifted from “improve what we have” to “design for the defined load and latency targets.”
Second, it matured their testing strategy. Unit tests were no longer enough. They introduced automated load testing into CI, failure injection scenarios, and performance regression thresholds. Test cases were no longer written to check correctness alone—they were written to validate the success criteria.
The most important realization was uncomfortable: until success was measurable, many of their design decisions were essentially random. They were choosing patterns based on familiarity, preference, or trend—not because those choices demonstrably supported a defined outcome.
Once success became measurable, architecture stopped being opinion-driven and became constraint-driven. And that shift elevated the quality of both their system and their engineering thinking.
Know Your Boundaries
Clarity isn’t just about knowing what you’re building. It’s also knowing what you’re not building.
Knowing what problems you are not going to solve:
- Prevents overengineering
- Reduces hidden expectations
- Protects timelines and focus
- Avoids accidental scope change
Saying “This system will not handle load above 10,000 requests per second” is more powerful than promising it will handle high load. Choose your boundaries consciously. Don’t try to solve everything.
Example
A team is building a reporting dashboard for business stakeholders.
During the planning stage, they start receiving requests to support real-time analytics and predictive insights for other users.
Instead of trying to accommodate everything, the team sets a clear boundary:
“This system will provide daily summary reports because that’s what the business stakeholders need. It will not support real-time analytics or forecasting.”
Because of this decision:
- The system stays simpler and more reliable
- Performance remains predictable
- Delivery timelines stay realistic
- Stakeholders understand what to expect, and what not to expect
Another team later builds a separate specialized analytics system without overloading the original one.
Boundaries don’t limit impact. They protect focus, reduce complexity, and prevent a good system from becoming an unfocused one.
Go Beyond the Problem Statement
Most requirements are one-dimensional. They revolve around the core problem statement. If you define success only within that boundary, you may technically succeed, but won’t make a lasting impact.
Success must extend beyond functional correctness. It must include qualities that preserve the system and the team.
Example
The problem statement: Build a configurable pricing engine where business users can define pricing rules.
A narrow definition of success would be:
- Business users can create rules.
- The system calculates correct prices.
- The system meets a specific performance criterion.
- The release happens on time.
But mature success definitions go further.
Success also means:
- A user should not take more than 3 minutes to add a new pricing rule.
- The UI should be intuitive enough for a new operator to understand it without having to read a 10-page training manual.
- A new support engineer should be able to understand the system in 2 days.
- Debugging a pricing discrepancy should not take more than 30 minutes.
- A new programmer should be able to take handover of the codebase in one week.
Notice what changed. We moved beyond “Does it work?”
We asked:
- Is it usable?
- Is it understandable?
- Is it diagnosable?
- Is it maintainable?
- Is it sustainable?
These are not secondary concerns. These are part of success.
Defining success only within the problem statement produces short-term wins. It rewards completion, not quality.
When defining success, ask:
“Beyond solving the problem, what must we do for this to be a good outcome for everyone involved?”
That question separates delivery from engineering.
When success is defined by outcomes rather than activity, engineering decisions become more meaningful. Priorities align with value. Trade-offs become clearer.
Defining success and assigning boundaries is what turns effort into meaningful progress.
CONTINUE THE JOURNEY
NEXT ARTICLE
Disciplined Execution
Good intentions create plans. Discipline creates results.
Continue Reading
Previous
Problem Framing
Next
Disciplined Execution
