Quick Summary – Feature Flags
A feature flag separates deploying code from exposing behavior to users. It lets you release gradually, observe the result, and disable a feature quickly when the metrics move in the wrong direction.
- Separate release steps — Deploying code and enabling a feature become two independently controlled changes.
- Roll out gradually — Start with internal users or a small percentage before expanding to all traffic.
- Use stable targeting — Hash a stable user or tenant identity so users do not switch variants between requests.
- Protect the fallback — Define what the application does when the flag provider is unavailable.
- Monitor changes — Record flag updates and watch errors, latency, resources, and business metrics together.
- Remove old flags — Give every short-lived flag an owner, review date, and cleanup plan.
A feature flag is a switch that controls application behavior without requiring another code deployment. You can ship code while keeping it disabled, expose it to a small and stable audience, watch the results, and turn it off when the evidence says the release is not ready.
Table of Contents
- The switch that separates shipping from releasing
- Four useful flag categories
- Design the flag before you need it
- A gradual rollout gives you room to stop
- Picking a flag storage model
- Make flag changes visible
- Old flags quietly become technical debt
- Tests must cover both sides
- What I check before production
- Before Enabling a Feature Flag, Check These Items
- Frequently Asked Questions
- Sources
The switch that separates shipping from releasing
I have watched a release go wrong because the only recovery option was another deployment. That is a slow way to react when users are already seeing the problem. A feature flag gives me a smaller control surface: the code can be present on the server while the behavior stays disabled or limited to a carefully chosen group.
A feature flag is a configuration mechanism that turns application behavior on or off without deploying the code again. You can expose a new feature to your team, one customer group, or a small percentage of traffic. If the metrics deteriorate, the first response can be a controlled configuration change instead of another build.
That separation matters. Deploying code and exposing behavior are different events.
Deployment, release, and rollback are not synonyms
A deployment loads new application code onto a server. A feature flag decides when and for whom that code runs. A rollback returns the application to an earlier version. Keeping those meanings separate makes incident decisions much less confusing.
| Concept | What changes? | How you revert it |
|---|---|---|
| Deployment | Application code on the server | Previous version or another deployment |
| Feature flag | The conditions under which a feature runs | Change the flag value |
| Rollback | The running application version | Return to the previous application version |
A feature flag does not replace a rollback. If the database schema changed, a data migration cannot be reversed, or the new code affects every request before the flag is checked, switching it off may not be enough.
What I record: the flag’s purpose, default value, owner, targeting rules, creation date, and removal or review date.
Four useful flag categories
Not every flag has the same lifespan or risk. A short-lived release control should not be managed like a subscription permission that may remain for years. I have seen temporary settings become permanent simply because nobody knew who owned them.
Release flags
A release flag keeps unfinished functionality in the production codebase while development continues. It stays off until testing is complete, then opens for a target group. Its lifespan should be short. Once the feature is fully released, remove the old code path and the flag record.
Short-lived. That is the point.
Experiment flags
An experiment flag shows different behavior to comparable user groups so you can measure the result. An A/B test is a common example. Use a stable evaluation key so a user does not move between groups on every request. Without a persistent user or anonymous visitor identifier, the experiment data quickly loses meaning.
Operational flags
An operational flag temporarily disables an expensive feature during heavy traffic, a third-party outage, or resource pressure. Report exports, recommendation engines, and image processing can fit this category. The operations team needs a fast way to change it, with access control and an audit trail.
Permission flags
A permission flag enables a feature based on a customer plan, role, or tenant setting. These flags often live for a long time and start to resemble an authorization system. Do not leave a security decision to a client-side flag. The server must still validate the user’s permissions for every protected resource.
Design the flag before you need it
A useful flag is more than a true or false value. At minimum, I keep its key, owner, purpose, creation date, targeting rules, and removal condition together.
Name the behavior, not just the project. new_ui looks convenient, but after a few months nobody may remember which screen or version it describes. billing_invoice_pdf_v2 carries more context. Treat a flag name like an API: changing the meaning of the same key in different services is an easy way to create a difficult bug.
Evaluate flags through one central application layer. If every controller or frontend component implements its own reading logic, defaults, cache durations, and failure behavior will drift apart.
if (flags.isEnabled("billing_invoice_pdf_v2", userContext)) {
return renderNewInvoicePdf();
}
return renderLegacyInvoicePdf();
Here, the decision stays outside the PDF generation code. The flags.isEnabled layer owns targeting, the default value, and communication with the provider.
Decide what happens when the flag service cannot be reached. false may be a safe default for a non-critical search feature. Authentication and payment flows need a deliberate fallback based on the business logic and failure mode.
What I require in code review: an explicit failure behavior for every flag, including what happens when its provider is unavailable.
Tip
Keep flag evaluation behind one application interface. That gives every caller the same defaults, targeting rules, cache behavior, and failure handling.
A gradual rollout gives you room to stop
The safest release is rarely a single jump to 100 percent. I start with the application team, then a small internal group, then a low-risk customer segment. Only after the numbers look healthy do I expand the audience.
- Start disabled: Deploy the new code with the flag off by default.
- Try internal users first: Use team accounts and test tenants as the first observation point.
- Open a small percentage: Start with 1 percent of users or traffic and use a random but stable assignment.
- Watch the metrics: Check error rate, latency, resource usage, and the relevant business metric together.
- Increase in steps: Leave an observation period between stages such as 1 percent, 5 percent, 25 percent, 50 percent, and 100 percent.
- Apply the stop condition: If a written threshold is exceeded, switch the flag off or return to the last safe percentage.
The percentages are not magic. Stable targeting and a clear stop condition matter more.
For percentage-based rollout, create a stable hash from the user or tenant identity instead of generating a new random number for every request. Otherwise the same user may see the new interface on one request and the old interface on the next.
bucket = stableHash(user.id + ":billing_invoice_pdf_v2") % 100
if bucket < rolloutPercentage:
enableFeature()
This small example is not a complete production safety layer. The hash algorithm, privacy of the identity input, tenant boundaries, and audit trail for flag changes need separate consideration.
If the server evaluates the flag, send only the result the client needs. Exposing the entire flag store to a browser can reveal future feature names and internal business rules.
Before rollout: write down the metrics and stop threshold for every stage.
Example
A rollout can move from 1 percent to 5 percent, 25 percent, 50 percent, and 100 percent, but only after an observation period and a written stop threshold for each stage.
Picking a flag storage model
Flag values can live in a versioned configuration file, a database, a dedicated service, or a third-party platform. I choose based on how quickly values must change, how many people manage them, how much access control is needed, and how much dependency the application can tolerate.
| Approach | Strength | Limitation |
|---|---|---|
| Configuration file | Simple, versioned, and has few external dependencies | A change usually requires a deployment |
| Database | Supports an external panel and detailed targeting | Schema, cache, and connection failures must be handled |
| Flag service | Gradual rollout, auditing, and team access are more organized | Adds a network dependency and operating work |
| Client-side setting | Fast for harmless interface experiments | Cannot be trusted for security or authorization |
OpenFeature documents a provider-based, vendor-neutral model that separates flag evaluation from application code. The application can keep the same evaluation interface while the source of flag data changes.
For a single application on a small VPS, a versioned YAML or JSON file may be enough. If you need changes without a deployment, reload the file safely, write it atomically, and keep the last valid configuration when new content is invalid.
features:
billing_invoice_pdf_v2:
enabled: false
rollout_percentage: 0
owner: billing-team
expires_at: 2026-12-31
I tested this pattern on a Proxmox VM by placing an invalid flag file where the service expected its configuration. The application kept its last valid values, which was the safer failure mode. My first version still lacked proper validation and atomic replacement, so I fixed both before using the pattern again.
Write new configuration to a temporary file, validate it, and replace the live file atomically. With several application instances, measure how long each instance takes to see the same value.
When you use a flag service, account for network latency and unavailability. A short-lived cache reduces load, but it also delays a disable decision reaching every instance.
From the field
I once put an invalid flag file on a Proxmox test VM. The application kept its last valid configuration, which was safer than failing open, but I still had to improve validation and atomic replacement before trusting the pattern.
Make flag changes visible
Changing a flag can alter user behavior immediately, even though no application code changed at that moment. I treat flag changes as production changes: restrict access, record them, and make their effects visible.
- Record the user who made the change.
- Store the old and new values in an audit log.
- Keep the change time in UTC.
- Record the flag key and targeting rule.
- Track evaluation failures in a separate metric.
- Keep sensitive user data out of log lines.
Logging every flag value on every request creates noise and cost. It is usually more useful to measure the effect on application metrics. Low-cardinality labels such as invoice_pdf_generation_total{variant="v2"} and invoice_pdf_errors_total{variant="v2"} can distinguish variants. An email address or raw user ID should not become a metric label.
After a change, I watch the 5xx rate, p95 or p99 latency, CPU, memory, database connection pool, and queue length. Clean application logs alone do not prove that the new behavior is healthy.
If a feature flag participates in a security control, test that the disabled state genuinely prevents unauthorized access. Hiding a button in the frontend is not access control. The API endpoint must validate authorization on the server for every request.
Useful dashboard view: put flag changes, application errors, resource metrics, and business metrics on the same timeline.
Old flags quietly become technical debt
When a flag is forgotten after being enabled, the code carries two behaviors for much longer than necessary. Tests multiply, nobody knows which branch is still required, and the next developer becomes understandably nervous about removing the old one.
GitLab’s development documentation also treats a flag’s lifecycle together with its type, owner, and removal plan. The practical lesson is simple: plan the removal when you create the flag.
A cleanup schedule that survives handover
- Set a review date when you create the flag.
- Measure whether the old code path is still used after release.
- Remove the old branch when the flag is 100 percent enabled and rollback is no longer needed.
- Update tests, documentation, and configuration records.
- Delete the flag from its store and keep an audit record.
Automatic deletion may be wrong for long-lived permission or operational flags. Review their ownership and access regularly instead. Short-lived release flags can be tracked with static analysis, pull request checks, or an expiration-date alert.
One rule I keep: no flag without an owner and a planned review date.
Tests must cover both sides
If new code is tested only with its flag enabled, the disabled branch can quietly decay. I write unit and integration tests for both behaviors.
FEATURE_BILLING_INVOICE_PDF_V2=false pytest tests/billing
FEATURE_BILLING_INVOICE_PDF_V2=true pytest tests/billing
This environment-variable example is a quick starting point for a small service. Do not mix feature values with secrets in production configuration: a setting that enables a feature is not an API key or a database password.
Make a flag change visible in the CI/CD pipeline as a separate change from the application deployment. Production enablement can use an approval step, two-person review, or an automated health check. Leave an observation window after enabling the flag so the next pipeline step does not amplify a problem.
Database migrations need particular care. If the new code can work with the old schema, add a backward-compatible schema first, deploy the code, enable the flag, and remove the old column or table last. Enabling the flag before running the migration can cause an outage, especially when several application instances are involved.
Ask the awkward question early: can I turn this flag off after the migration? If the answer is no, the migration and recovery plan need another review.
What I check before production
A feature flag reduces release risk; it does not make risk disappear. While both paths are active, I check data consistency, cache behavior, queues, third-party API limits, and the recovery path.
If a new interface is cached by a CDN, content that varies by user segment must be represented correctly in the cache key. The caching details in What Is a CDN? How It Speeds Up Websites? are useful here. On the server side, the configuration validation habits from How to Install and Configure Nginx on Ubuntu provide another safe check, independent of the flag itself.
A VPS snapshot can provide a fast recovery option during a release, but it does not by itself guarantee database consistency or preserve the flag change as an application-level rollback plan. The limits described in What Is a VPS Snapshot? A Guide to Point-in-Time Backups still apply. Track flag definitions and removal changes together with What Is Git? A Practical Guide to Version Control so the release history remains understandable.
My final staging test: exercise the old path, new path, data migration, monitoring, flag-provider failure, and recovery plan together before enabling production traffic.
Caution
A flag is not a rollback and a client-side switch is not authorization. Database migrations and server-side permission checks still need their own safety measures.
Before Enabling a Feature Flag, Check These Items
- Name the flag after the behavior it controls.
- Assign an owner and an expiration or review date.
- Define the default value and provider failure behavior.
- Test both the enabled and disabled code paths.
- Choose stable user or tenant targeting for percentage rollouts.
- Set metrics and a stop threshold for every rollout stage.
- Prepare the cleanup and recovery plan before enabling production traffic.
Start with one small, observable release flag rather than building a large platform on day one. Give it an owner and a removal date before the first user ever sees the new behavior.
When I’m rolling out a WordPress change that requires a brief service pause, I use our guide on how to enable maintenance mode in WordPress to choose the safest approach and plan the rollback.
Frequently Asked Questions
Can a feature flag replace a deployment rollback?
No. A flag can disable behavior while the deployed code remains on the server, but it cannot undo an incompatible schema change, repair corrupted data, or restore every part of a previous application version. Use flags for controlled exposure and rollbacks for version recovery. For important releases, test both mechanisms instead of treating one as a substitute for the other.
Should feature flags be evaluated on the client or server?
Use server-side evaluation for permissions, sensitive business rules, and anything that must not be bypassed. Client-side evaluation can be convenient for harmless interface experiments, but browser code is visible and can be modified. Even when the frontend hides a control, the API must enforce authorization independently on every protected request.
How should I choose a percentage rollout?
Start with a small audience that you can observe, often internal users or a low-risk segment, then increase in deliberate stages. The exact percentages matter less than stable assignment, useful metrics, and a stop condition. A user or tenant should stay in the same variant during the rollout, so use a stable hash rather than a new random choice on every request.
What happens if the feature flag service goes down?
The application should have an explicit failure policy for each flag. A non-critical feature may safely default to disabled, while an authentication or payment flow may require a carefully designed fallback. Cache values can reduce dependency on the flag service, but they delay changes. Test outages so the chosen behavior is known rather than accidental.
How long should a feature flag remain in the codebase?
Release flags should usually be short-lived: once the feature is fully enabled and rollback is no longer needed, remove the old branch and the flag. Permission and operational flags may be long-lived, but they still need an owner, access review, documentation, and an audit trail. An expiration or review date prevents temporary controls from becoming invisible permanent behavior.
Are feature flags safe for authorization?
A flag can help select an authorized product capability, but it should not be the only security boundary. The server must check the authenticated user, role, tenant, and resource permissions for each request. Never trust a browser-provided flag value. Treat the flag as one input to authorization logic, not as proof that the caller is allowed to perform an action.
Sources
- OWASP – Authorization Cheat Sheet — cheatsheetseries.owasp.org