bindbox-game/.planning/research/SUMMARY.md

Project Research Summary

Project: Bindbox Game — Profit/Loss Analytics Service Layer Domain: Go/GORM/MySQL financial aggregation functions for a game/e-commerce platform Researched: 2026-03-21 Confidence: HIGH

Executive Summary

This milestone implements two reusable service-layer functions — QueryUserProfitLoss and QueryActivityProfitLoss — that aggregate platform-perspective profit and loss data across user and activity dimensions. The domain is internal financial analytics on an existing Go 1.24 / GORM 1.25 / MySQL 8.x stack. No new runtime dependencies are required. All necessary shared logic (game-pass classification, prize cost with multiplier, profit computation) already exists in internal/service/finance/profit_metrics.go and must be reused without reimplementation. The key architectural decision is a dedicated internal/service/finance/ package with a clean Service interface, constructor accepting only DbR (read replica), and typed input/output structs — not an extension of existing HTTP handler logic.

The recommended query approach is the fan-out + in-memory merge pattern already established in the codebase: issue separate, independently scoped db.Table(...).Scan() calls per data source (orders, inventory, draw logs, ledger), then merge results in Go using a map[int64]*ProfitLossResult. This avoids Cartesian products from multi-table JOINs, keeps individual queries testable in isolation, and remains compatible with the SQLite test harness. Raw SQL (db.Raw()) should be used only when the query requires more than two levels of subqueries, consistent with the existing codebase convention.

The highest-severity risks are revenue double-counting (when one order spans multiple activities), silent scan errors (GORM's Scan() does not surface type-mismatch failures), and misclassifying game-pass orders as zero-revenue orders. All three have prior-art evidence in the existing dashboard code, with fix patterns already established. The new service layer must enforce: CAST(... AS SIGNED) on any SUM containing division, strict mutual exclusion between game-pass and cash revenue paths, refunded-order exclusion from both revenue and cost, and error propagation from every Scan() call.

---

Key Findings

Recommended Stack

The existing stack is fully sufficient. Go 1.24, GORM 1.25 with gorm.io/gen v0.3.26, MySQL 8.x with read/write split via gorm.io/plugin/dbresolver, go.uber.org/zap (wrapped as logger.CustomLogger), testify for assertions, and in-memory SQLite via NewSQLiteRepoForTest() for integration tests. Adding no new dependencies reduces risk and keeps the codebase coherent.

Core technologies:

• Go 1.24: Primary language — existing toolchain, no change
• GORM 1.25: ORM — use db.Table().Select().Scan() (Style A) for single-dimension GROUP BY; db.Raw().Scan() (Style B) for multi-level subqueries
• MySQL 8.x (DbR): Read replica — all analytics queries must route here via repo.GetDbR()
• logger.CustomLogger: Project-standard logger — inject at constructor, not package-level
• SQLite (test only): In-memory test DB — NewSQLiteRepoForTest(); note SQLite does not support CAST(AS SIGNED) or GREATEST() — abstract these into Go helpers
• Existing finance.* utilities: ClassifyOrderSpending, IsGamePassOrder, ComputeGamePassValue, ComputePrizeCostWithMultiplier, ComputeProfit, NormalizeMultiplierX1000 — all must be called, never re-derived

Expected Features

Must have (table stakes — P1):

• Revenue calculation: actual_amount + discount_amount (coupon discount adds back real value) with strict refund/void exclusion
• Game-pass order classification via finance.IsGamePassOrder — three-condition detection, mutual exclusion from cash revenue
• Game-pass value derivation: draw_count × activity_price via finance.ComputeGamePassValue
• Prize cost with item-card multiplier via finance.ComputePrizeCostWithMultiplier
• Profit calculation via finance.ComputeProfit returning (int64, float64)
• Time-range filter: *time.Time start/end, nil means no bound — never use zero-value sentinel
• User-dimension aggregation: QueryUserProfitLoss(ctx, ProfitLossParams) accepting []int64 user IDs (empty = all users)
• Activity-dimension aggregation: QueryActivityProfitLoss(ctx, ProfitLossParams) accepting one activity ID
• ProfitLossResult struct: total revenue, cost, profit, profit_rate, plus []ProfitLossBreakdown
• Canonical AssetType enum: Points (1), Coupon (2), ItemCard (3), Product (4), Fragment (5), All (0)
• All monetary values stored as int64 fen — never float64 for storage
• Read-only DB routing: constructor injects repo.GetDbR() only — GetDbW() must not appear anywhere in this package
• Error propagation: every Scan() error checked and returned, never swallowed

Should have (differentiators — P2):

• Per-asset-type cost breakdown (5 types as separate breakdown slice entries)
• Fragment asset type cost integration via fragment_synthesis_logs
• Batch activity IDs support ([]int64 activity IDs, not just one)

Defer (v2+):

• Redis TTL caching wrapper around the query functions (defer until query latency exceeds 2s)
• Incremental / time-bucketed aggregation with materialized stats tables (requires schema additions)
• Douyin (livestream) order integration into user-dimension function

Architecture Approach

Note: A separate ARCHITECTURE.md was not produced; architecture findings are synthesized from STACK.md and codebase analysis embedded in all three research files.

The architecture follows the established layered pattern: new package internal/service/finance/ with a Service interface and constructor accepting logger.CustomLogger and mysql.Repo. Business logic lives exclusively in service functions; HTTP handlers call service functions and handle pagination, auth, and response formatting. The fan-out query pattern (multiple targeted Scan() calls merged in Go) replaces any attempt at a single mega-JOIN. Pure finance computation functions (no DB access) remain in profit_metrics.go; new DB-querying logic lives in separate, focused files.

Major components:

1. service.go — Service interface definition + New(logger, repo) constructor; stores dbR *gorm.DB (read-only handle) and logger
2. params.go — AssetType constants, UserProfitLossParams, ActivityProfitLossParams, ProfitLossResult, ProfitLossBreakdown types; shared between both query files
3. query_user.go — QueryUserProfitLoss implementation: ID collection, fan-out scans, in-memory merge calling existing finance.* utilities
4. query_activity.go — QueryActivityProfitLoss implementation: same pattern, activity-dimension scoping with proportional revenue attribution
5. profit_metrics.go (existing) — pure business logic functions; no modification needed
6. service_test.go — integration tests using NewSQLiteRepoForTest(); covers boundary cases (zero revenue, refunded orders, game-pass, legacy order_id=0 inventory)

Critical Pitfalls

1. MySQL SUM with division returns Decimal, not SIGNED integer — Wrap every SUM(... / ...) expression with CAST(... AS SIGNED) in SQL. Scanning Decimal into int64 silently returns 0. Already hit in dashboard_activity.go:174; the fix is established — apply it consistently.

2. Revenue double-counting when one order spans multiple activities — Use the two-level subquery attribution pattern from dashboard_activity.go:197-212: compute draw_count per (order, activity) and total_count per order in separate derived tables, then prorate: actual_amount * draw_count / total_count. Naive SUM(actual_amount) grouped by activity fans out the full order to every matching activity.

3. Game-pass orders misclassified as zero-revenue orders — Use strict mutual exclusion: if IsGamePassOrder() returns true, revenue = draw_count × activity_price; otherwise revenue = actual_amount + discount_amount. Never sum both paths together. Three detection conditions must all be checked — not just source_type=4.

4. Silently ignored Scan() errors causing all-zero results — Every Scan() call must check .Error and return the error to the caller. "All zeros" is indistinguishable from a failed query without this check. This pattern is missing from existing dashboard code and must be corrected in the new package.

5. Refunded order inventory counted as prize cost — Always join user_inventory to orders on order_id and filter orders.status = 2, with the legacy escape hatch OR user_inventory.order_id = 0 OR user_inventory.order_id IS NULL. Add NOT LIKE '%void%' on user_inventory.remark. Refunds update orders.status but do not delete inventory rows.

6. Writing analytics queries to the write DB (DbW) — The constructor must accept and store only repo.GetDbR(). No call to GetDbW() should exist in internal/service/finance/. Enforce via grep in CI.

---

Implications for Roadmap

Based on combined research, a 3-phase structure is recommended. All P1 features are tightly interdependent (revenue depends on game-pass classification, cost depends on multiplier logic, profit depends on both) so they belong in one implementation phase. Per-asset-type breakdown is isolated enough to be a second phase. Testing and hardening constitute a third phase.

Phase 1: Foundation and Core P&L Functions

Rationale: All P1 features share the same data sources and query infrastructure. Building them together ensures the fan-out pattern, error handling convention, and read-replica routing are established consistently from the start. Deferring any P1 feature creates inconsistency in the result struct and breaks the ProfitLossResult contract for callers.

Delivers: Working QueryUserProfitLoss and QueryActivityProfitLoss with correct revenue (cash + game-pass), correct cost (with multiplier), correct profit, time-range filter, and refund/void exclusion. New package skeleton: service.go, params.go, query_user.go, query_activity.go.

Addresses (from FEATURES.md): All P1 features — revenue calculation, game-pass classification/derivation, prize cost with multiplier, profit formula, time-range filter, user-dimension aggregation, activity-dimension aggregation, composable filter struct, result type, AssetType enum, multi-user batch, read-DB enforcement.

Avoids (from PITFALLS.md): Decimal/int64 scan mismatch (CAST), revenue double-counting (subquery attribution), game-pass mutual exclusion, write-DB usage, silently swallowed scan errors.

Research flag: Standard patterns — established codebase conventions are documented; no additional research phase needed.

Phase 2: Per-Asset-Type Breakdown

Rationale: The breakdown slice in ProfitLossResult can be populated as a separate set of GROUP BY legs per asset type once the core aggregation is proven correct. This requires extending the SQL or adding additional scan passes — but must not alter the top-level totals, making it safe to do independently.

Delivers: Populated []ProfitLossBreakdown in the result, with one entry per AssetType (Points, Coupon, ItemCard, Product, Fragment). Fragment cost integration from fragment_synthesis_logs.

Addresses (from FEATURES.md): Per-asset-type breakdown (P2), Fragment synthesis cost (P2).

Avoids (from PITFALLS.md): Missing asset type silently understating cost ("Looks Done But Isn't" checklist item).

Research flag: Needs shallow research — Fragment synthesis log schema and join path require verification against the current DB model before implementation.

Phase 3: Batch Activity IDs and Hardening

Rationale: Batch activity ID support ([]int64) is a low-complexity extension once the single-activity path is correct. Hardening (additional test cases, CI grep gate, load test) consolidates correctness guarantees.

Delivers: QueryActivityProfitLoss accepting []int64 activity IDs. CI enforcement of GetDbW absence. Integration tests covering all "Looks Done But Isn't" checklist items. Load test verification with 1,000 activities.

Addresses (from FEATURES.md): Batch activity IDs support (P2).

Avoids (from PITFALLS.md): Empty []int64{} producing invalid SQL WHERE IN (); performance trap of unbounded activity fetch; missing LIMIT guard.

Research flag: Standard patterns — no research phase needed.

Phase Ordering Rationale

• Phase 1 must precede Phase 2 because the ProfitLossResult struct and fan-out pattern must be stable before extending it with per-type breakdown legs.
• Phase 2 must precede Phase 3 because batch activity ID support needs the full result struct (including breakdown) to be defined first.
• The proportional revenue attribution pattern (subquery join) is the most complex SQL in Phase 1 and must be designed before any other query is written — it anchors the activity-dimension function's correctness.
• SQLite test compatibility limits matter for Phase 1: CAST(AS SIGNED) and GREATEST() must be abstracted into Go helpers or test-specific SQL variants before the integration test suite is written.

Research Flags

Phases needing deeper research during planning:

• Phase 2: Fragment synthesis log schema — verify fragment_synthesis_logs table columns, join path to user_inventory or activity_id, and whether the cost model for fragments differs from physical goods.

Phases with standard patterns (skip research-phase):

• Phase 1: All patterns are documented in the existing codebase with explicit prior-art examples.
• Phase 3: Batch ID extension and CI hardening are mechanical changes on established patterns.

---

Confidence Assessment

Area	Confidence	Notes
Stack	HIGH	All findings verified directly from go.mod, existing source files, and inline code comments — no speculation
Features	HIGH	Derived from direct codebase analysis of existing dashboard implementations and profit_metrics.go; requirements validated against PROJECT.md
Architecture	HIGH	Architecture inferred from STACK.md (no separate ARCHITECTURE.md produced); all patterns confirmed from multiple existing service examples in codebase
Pitfalls	HIGH	Every pitfall has direct evidence — code comments, inline bug fixes, or .planning/codebase/CONCERNS.md entries in the existing codebase

Overall confidence: HIGH

Gaps to Address

• ARCHITECTURE.md was not produced by the parallel research phase. Architecture guidance was successfully recovered from STACK.md (which contained the service constructor pattern, file structure, and query patterns) and from codebase analysis referenced in FEATURES.md and PITFALLS.md. No meaningful gap results — all architectural decisions are documented in this summary.

• Fragment asset type cost model is undefined for v1. The fragment_synthesis_logs table exists and the AssetType enum entry is defined, but the exact cost calculation formula and join path are not yet verified. Address in Phase 2 planning with a focused schema review.

• SQLite test compatibility: CAST(AS SIGNED), GREATEST(), and LIKE 'GP%' are MySQL-specific. Integration tests on SQLite will require either Go-layer abstraction of these expressions or conditional SQL paths. This is a known constraint; address during Phase 1 test writing, not a blocker.

• Batch activity IDs deferred to Phase 3. The current QueryActivityProfitLoss design assumes one activity ID. The parameter struct should use []int64 from the start (even if Phase 1 only enforces len(activityIDs) == 1) to avoid a breaking interface change in Phase 3.

---

Sources

Primary (HIGH confidence — direct codebase analysis)

• internal/service/finance/profit_metrics.go — existing shared finance primitives; IsGamePassOrder, ComputeProfit, ComputePrizeCostWithMultiplier, ClassifyOrderSpending
• internal/api/admin/dashboard_activity.go — activity-dimension aggregation, prior-art bug fixes for Decimal/int64, double-counting, game-pass classification (lines 146-274)
• internal/api/admin/dashboard_spending.go — user-dimension aggregation, multi-join fan-out pattern
• internal/api/admin/dashboard_user_spending.go — per-user spending drill-down
• internal/repository/mysql/mysql.go — Repo interface, GetDbR() / GetDbW() split
• internal/repository/mysql/testrepo_sqlite.go — NewSQLiteRepoForTest() pattern
• internal/service/user/user.go — canonical Service interface + constructor pattern
• .planning/codebase/CONCERNS.md — flagged 113 GetDbW() calls in handler layer; silently swallowed errors in financial paths
• go.mod — confirmed dependency versions (Go 1.24.0, GORM 1.25.9, testify 1.11.1)

Secondary (HIGH confidence — official documentation cross-referenced with codebase evidence)

• GORM v1.25 docs: soft-delete not auto-injected into raw JOIN strings — confirmed by existing code comments
• MySQL 8.x docs: SUM() with division promotes to Decimal — confirmed by dashboard_activity.go:174 comment and CAST(AS SIGNED) fix pattern

---

Research completed: 2026-03-21 Ready for roadmap: yes