AGENT SKILLBY ERONRED
A/B test store listing
AT A GLANCE
A/B test store listing is an agent skill for CRO. When the user wants to A/B test App Store product page elements to improve conversion rate. The eronred/aso-skills repository on GitHub, maintained by eronred, has 1.8K stars and the MIT license. To add it, run npx skills add eronred/aso-skills --skill ab-test-store-listing
- SOURCE REPOSITORY
- eronred/aso-skills (opens GitHub in a new tab)
- REPO STARS
- SOURCE REPOSITORY
- LICENSE
- MIT
- CATEGORIES
- SKILL.MD SIZE
- 6.8 KB214 LINES
QUICK ANSWER
- WHAT IT IS01
- When the user wants to A/B test App Store product page elements to improve conversion rate. Its edge is a particular angle on A/B test store listing, giving the agent tighter constraints than a plain A/B test store listing request.
- INPUTS02
- Context the agent needs: your goals, audience, constraints, and any source material the skill asks for.
- OUTPUT03
- A ready-to-use result: the analysis, copy, or recommendations the agent produces.
INSTALL
One command adds this skill from its source repository.
npx skills add eronred/aso-skills --skill ab-test-store-listingREAD THE SKILL.MD BELOW BEFORE YOU RUN IT. SKILLS INSTRUCT YOUR AGENT.
SKILL.MD
The full file, as published in the source repository. Copy it, download it or open it on GitHub.
ON THIS PAGE · 22
A/B Test Store Listing#
You are an expert in App Store product page optimization and A/B testing. Your goal is to help the user design, run, and interpret tests that improve their App Store conversion rate.
Initial Assessment#
- Check for
app-marketing-context.md— read it for context - Ask for the App ID
- Ask for current conversion rate (if known from App Store Connect)
- Ask for daily impressions (determines test duration)
- Ask: What do you want to test? (icon, screenshots, description, etc.)
What You Can Test#
Apple Product Page Optimization (PPO)#
Apple's native A/B testing tool in App Store Connect.
| Element | Testable? | Notes |
|---|---|---|
| App icon | Yes | Up to 3 variants |
| Screenshots | Yes | Up to 3 variants |
| App preview video | Yes | Up to 3 variants |
| Description | No | Not testable via PPO |
| Title | No | Not testable via PPO |
| Subtitle | No | Not testable via PPO |
Limitations:
- Only tests against organic App Store traffic
- Minimum 90% confidence required to declare winner
- Tests run for 7-90 days
- Can only run one test at a time
- Traffic split is automatic (not configurable)
Custom Product Pages (CPP)#
35 custom product pages per app, each with unique:
- Screenshots
- App preview videos
- Promotional text
Use for:
- Different audiences (from different ad campaigns)
- Different value propositions
- Seasonal messaging
- Localized creative for specific markets
Not a true A/B test — CPPs are targeted pages linked from specific URLs/campaigns, not random traffic splits.
Test Prioritization#
Impact × Effort Matrix#
| Element | Impact on CVR | Effort | Priority |
|---|---|---|---|
| First screenshot | Very High (15-30% lift possible) | Medium | 1 |
| App icon | High (10-20% lift possible) | Medium | 2 |
| Screenshot order | Medium (5-15% lift possible) | Low | 3 |
| Screenshot style | Medium (5-15% lift possible) | High | 4 |
| Preview video | Medium (5-10% lift possible) | High | 5 |
What to Test First#
Always start with the first screenshot. It has the highest impact because:
- It's the first thing users see in search results
- 80% of users never scroll past the first 3 screenshots
- Small improvements here affect every visitor
Test Design Framework#
Step 1: Hypothesis#
Write a clear hypothesis before each test:
If we [change], then [metric] will [improve/increase] because [reason].Examples:
- "If we add social proof ('5M+ users') to the first screenshot, conversion rate will increase because it builds trust"
- "If we change the icon from blue to orange, tap-through rate will increase because it stands out more in search results"
- "If we show the app's AI feature first instead of the basic editor, conversion will increase because AI is the key differentiator"
Step 2: Variants#
Design 2-3 variants (including control):
| Variant | Description | Hypothesis |
|---|---|---|
| Control (A) | Current version | Baseline |
| Variant B | [specific change] | [why it might win] |
| Variant C | [different change] | [why it might win] |
Rules for good variants:
- Change ONE thing per test (isolate the variable)
- Make the change significant enough to detect (don't test subtle color shifts)
- Each variant should have a clear hypothesis
- Don't test more than 3 variants (dilutes traffic)
Step 3: Sample Size#
Calculate required test duration:
Daily impressions: [N]
Current conversion rate: [X]%
Minimum detectable effect: [Y]% (relative improvement)
Confidence level: 95%
Required sample per variant: ~[N] impressions
Estimated duration: [N] daysRules of thumb:
- < 1000 daily impressions: Tests take 30-90 days (consider if worth it)
- 1000-5000 daily impressions: Tests take 14-30 days
- 5000+ daily impressions: Tests take 7-14 days
- Need at least 1000 impressions per variant for meaningful results
Step 4: Run the Test#
In App Store Connect:
- Go to Product Page Optimization
- Create a new test
- Upload variant assets
- Set test duration (recommend: let it run until statistical significance)
- Monitor but don't stop early
Step 5: Interpret Results#
Statistical significance:
- Apple requires 90% confidence minimum
- Aim for 95% confidence before making decisions
- Look at the confidence interval, not just the point estimate
What to look for:
- Conversion rate lift (primary metric)
- Impression-to-tap rate (for icon tests)
- Download rate (for screenshot/video tests)
- Segment differences (new vs returning, country, source)
Common Test Ideas#
Icon Tests#
| Test | Control | Variant | Expected Impact |
|---|---|---|---|
| Color | Current color | Contrasting color | 5-20% TTR change |
| Style | Detailed | Simplified | 5-15% TTR change |
| Element | Current symbol | Different symbol | 5-20% TTR change |
| Background | Solid | Gradient | 3-10% TTR change |
Screenshot Tests#
| Test | Control | Variant | Expected Impact |
|---|---|---|---|
| First screenshot | Feature-focused | Benefit-focused | 10-30% CVR change |
| Social proof | No social proof | "5M+ users" badge | 5-15% CVR change |
| Text size | Small text | Large, bold text | 5-10% CVR change |
| Style | Light mode | Dark mode | 5-15% CVR change |
| Layout | Device frame | Full-bleed | 5-10% CVR change |
| Order | Current order | Reordered by benefit | 5-15% CVR change |
Video Tests#
| Test | Control | Variant | Expected Impact |
|---|---|---|---|
| Has video | No video | 15s feature demo | 5-15% CVR change |
| Hook | Feature demo | Problem/solution | 5-10% CVR change |
| Length | 30s | 15s | 3-8% CVR change |
Output Format#
Test Plan#
Test Name: [descriptive name]
Element: [icon / screenshots / video]
Hypothesis: If we [change], then [metric] will [improve] because [reason]
Variants:
- Control (A): [description]
- Variant B: [description]
- Variant C: [description] (optional)
Estimated Duration: [N] days
Required Impressions: [N] per variant
Success Metric: [conversion rate / tap-through rate]
Minimum Detectable Effect: [X]%Test Results Interpretation#
When the user shares results:
- Is it statistically significant? (confidence level)
- What's the actual lift? (with confidence interval)
- Are there segment differences?
- What's the next test to run?
- Estimated annual impact (downloads × lift)
Testing Roadmap#
Provide a 3-month testing calendar:
- Month 1: [highest impact test]
- Month 2: [second priority test]
- Month 3: [third priority test]
Related Skills#
screenshot-optimization— Design screenshot variantsmetadata-optimization— Optimize non-testable elementsapp-analytics— Track conversion metricsaso-audit— Identify what to test first
FAQ
07 QUESTIONS
01What is the A/B test store listing agent skill?
When the user wants to A/B test App Store product page elements to improve conversion rate. It is listed under CRO in the 100xLab agent skills directory.
02How do I install A/B test store listing in Cursor, Claude Code, or Codex?
Run npx skills add eronred/aso-skills --skill ab-test-store-listing in the project where you want it, then ask your agent for the skill by name. The --skill flag installs only A/B test store listing, not every skill in the repository.
03What does the A/B test store listing SKILL.md cover?
The SKILL.md includes sections on Initial Assessment, What You Can Test, Apple Product Page Optimization (PPO), Custom Product Pages (CPP), Test Prioritization and Impact × Effort Matrix. Read the full text on this page or in plain markdown at /resources/agent-skills/eronred-aso-skills--ab-test-store-listing--jabqt6/raw.
04Which agents can use A/B test store listing?
A/B test store listing installs with the cross-agent skills installer (npx skills add), which sets it up for agents such as Claude Code, Cursor and Codex. Each agent loads the SKILL.md when you ask for the skill by name.
05What license is A/B test store listing under?
The eronred/aso-skills repository declares the MIT license. Read the license file in the repository before you copy or redistribute the skill.
06Is A/B test store listing free to use?
Reading, copying and installing A/B test store listing costs nothing: it is published in a public GitHub repository under the MIT license. Reuse and redistribution follow that license.
07Where is the source of A/B test store listing?
A/B test store listing comes from https://github.com/eronred/aso-skills, which has 1.8K GitHub stars. The SKILL.md is at https://github.com/eronred/aso-skills/blob/HEAD/skills/ab-test-store-listing/SKILL.md. The author is eronred.
Need an agent workflow built around your stack?
Tell us the workflow you want automated. We reply within 4 working hours (IST).
ATTRIBUTION
A/B test store listing was written by its original authors and is published in eronred/aso-skills (opens GitHub in a new tab) under the MIT license. All rights stay with the original authors.
100xLab did not write, maintain or endorse this skill. This page lists it for discovery. Repo stars belong to the source repository, not to the skill. Review any skill before you install it.