Complete specification for rewriting Swarmops v5 as using.cash v6: - Proposal: scope buckets (Working Rewrite / First Beyond Working / Maybe), non-goals (governance, geography, memberships permanently cut) - Design: 18 architectural decisions including preserved patterns (Basic/Logic, stored procs, credential split), new patterns (repository classes, chain abstraction, rate providers), and operational concerns (Galera awareness, continuation transactions, VAT booking, OOD denormalization) - Specs: 17 capability specifications with SHALL/MUST requirements and WHEN/THEN scenarios, covering accounting core, AP/AR, crypto treasury, exchange rates, organizations, users/auth, documents, comms, dashboard, end-of-month, backend worker, event bus, data migration, API layer, frontend app, and CI/packaging - Tasks: 145 implementation tasks grouped by subsystem, ordered by dependency (migration gates API/frontend work) Two rounds of adversarial review incorporated. Made-with: Cursor
25 KiB
Context
Swarmops v5 is a monolithic C# application targeting .NET Framework 4.5.2 on Mono. It comprises ~1,200 .cs files across 13 projects, connected by a hand-rolled ADO.NET data access layer (SwarmDb partial class) against MySQL with ~100 stored procedures. The frontend is ASP.NET WebForms (109 .aspx pages) with jQuery/EasyUI. The build pipeline (CruiseControl.NET, Debian packaging) has been broken for years.
The codebase carries twenty years of accumulated features from its origins as a political party membership register. Roughly half the domain surface (governance, geography, memberships, activists) is no longer relevant to its current purpose as a crypto-centric accounting system.
Stakeholders: the project owner (sole maintainer), existing organizations whose closed accounting ledgers must remain legally presentation-immutable, and tax authorities who may audit those ledgers.
Goals / Non-Goals
Goals:
- Build and run on .NET 8+ without Mono
- Replace ASP.NET WebForms frontend with React + Material UI SPA
- Clean database schema via ETL migration (v5 DB untouched)
- Strip all non-accounting features (governance, geography, memberships)
- Rename all namespaces, assemblies, and branding from Swarmops to UsingCash
- Preserve the financial core with its proven patterns intact
- Dockerized deployment with Gitea Actions CI
- Closed-ledger rendering parity: v6 must present identical numbers to v5
Non-Goals:
- Changing the database engine (MySQL stays)
- Rewriting the stored procedure security model (read/write/admin split preserved)
- Multi-chain crypto support in Bucket 1 (BCH only, but abstracted)
- Cloud-native storage or K8s in Bucket 1 (deferred to Bucket 2)
- Payroll, bank integration, or VAT reporting UI in Bucket 1 (deferred; VAT booking mechanics ARE in Bucket 1)
- Using
decimalfor intermediate currency calculations (deliberate:doubleis preserved from v5 to avoid hard-to-debug precision issues; documented as intentional trade-off)
Decisions
D1: Preserve Basic/Logic two-tier inheritance
Decision: Keep the BasicFoo (DTO) / Foo (rich domain) inheritance pattern and the PluralBase<TPlural, TSingular, TBasic> collection infrastructure.
Rationale: This pattern exists for a structural reason — it keeps the Database layer independent of Logic, which was essential during the MSSQL→MySQL migration and remains a clean separation boundary. The PluralBase generic eliminates duplication of complex hydration logic across dozens of entity types. Rewriting to a different pattern (e.g., AutoMapper + repositories) would be a large effort with no functional gain.
Alternatives considered:
- EF Core with POCO entities: would require abandoning the stored procedure write model and the credential split — too much lost for Bucket 1.
- Dapper with manual mapping: similar to current approach but loses the Basic/Logic split. No clear win.
Adaptation for v6: The SingularFactory uses reflection to find Logic subclasses. This is acceptable but should be initialized at startup (warm cache) rather than lazily discovered per-request.
D2: Break SwarmDb into per-domain repository classes
Decision: Replace the monolithic SwarmDb partial class with domain-scoped repository classes (e.g., FinancialRepository, UserRepository, OrganizationRepository) that share a common DatabaseConnection factory providing the read/write/admin credential split.
Rationale: The original author wanted one file per table with a shared GetDatabaseFor*() method. The partial class achieved this but made the class unboundedly large. Per-domain repositories achieve the same readability while being independently testable and navigable.
Pattern:
DatabaseConnection (shared factory)
├── GetForReading() → MySQL user with SELECT only
├── GetForWriting() → MySQL user with EXECUTE on write procedures
└── GetForAdmin() → MySQL user with DDL privileges
FinancialRepository : RepositoryBase
├── GetTransaction(id) — uses GetForReading()
├── CreateTransaction(...) — uses GetForWriting(), calls stored proc
└── GetAccountsForOrganization(orgId)
D3: ASP.NET Core minimal API as the HTTP layer
Decision: Use ASP.NET Core minimal API endpoints (not controllers) to replace the WebForms page methods and Json-*.aspx endpoints. WebSocket support via ASP.NET Core's built-in middleware replaces websocket-sharp.
Rationale: Minimal APIs are lighter weight, map naturally to the current flat endpoint structure (each Json-*.aspx becomes an endpoint group), and avoid the ceremony of MVC controllers for what is essentially a JSON API serving a React SPA.
Alternatives considered:
- ASP.NET Core MVC controllers: more structure than needed; the API is a thin translation layer over the Logic layer.
- gRPC: overkill for a browser-facing SPA; would require grpc-web proxy.
D4: React + Material UI + Vite frontend
Decision: Single-page application using React 18+, Material UI (MUI) component library, TypeScript, and Vite as the build tool.
Rationale: MUI provides a comprehensive component set (data grids, forms, dialogs, navigation) that maps well to the existing EasyUI-based UI. Vite provides fast builds and HMR. TypeScript catches errors that the current loosely-typed jQuery code misses.
The frontend communicates with the backend exclusively through the REST API and WebSocket — no server-side rendering, no ASPX page lifecycle.
D5: Clean v6 schema via ETL migration
Decision: v6 creates its own MySQL database with a clean schema. A migration tool reads the v5 database and populates v6 tables. The v5 database is never modified.
Rationale: The v5 schema carries 76 incremental upgrades, unused columns from cut features (geography, governance, memberships), naming inconsistencies, and stored procedures for deleted functionality. A clean schema lets v6 start with only the tables and procedures it needs.
Migration tool responsibilities:
- Read v5
FinancialAccounts,FinancialTransactions,FinancialTransactionRows, tags, budgets → write to v6 equivalents - Read v5
People(subset needed as Users) → write to v6Users - Read v5
Organizations→ write to v6Organizations - Read v5
Documentsmetadata → write to v6Documents(file storage paths preserved or remapped) - Read v5
Currencies,CurrencyExchangeRates→ write to v6 equivalents - Read v5 crypto tables (
HotBitcoinAddresses,HotBitcoinAddressUnspents) → write to v6 equivalents - Read v5
ExpenseClaims,InboundInvoices,OutboundInvoices,Payouts,Payments→ write to v6 equivalents - Mark migrated transactions in closed fiscal years as immutable in v6 schema
- Verification pass: render balance sheets and P&L for all closed periods from both v5 and v6, diff the output — must be identical
What is NOT migrated: governance tables, geography hierarchy, memberships, activists, volunteers, parleys, media, ballots, motions, polls, PirateWeb/Activizr-era data, unused optional data types.
D6: Closed-period immutability in v6 schema
Decision: v6 schema includes a ClosedThroughYear column on Organizations (replacing the v5 ObjectOptionalData pattern for OrgBooksClosedForYear). Transactions, transaction rows, and account structures within closed periods are protected by application-level enforcement (same as v5) plus a SQL trigger as a safety net.
Rationale: v5 relies entirely on application-level enforcement with no database-level guard. This is fragile — a bug in any code path could corrupt legally frozen books. Adding a SQL trigger that rejects INSERT/UPDATE/DELETE on FinancialTransactionRows where the transaction date falls within a closed period provides defense-in-depth without changing the application model.
D7: Instance-level RBAC
Decision: Implement role-based access control at the instance level for Bucket 1. Users are assigned roles; roles have permissions. The v5 Permission enum is preserved (trimmed of governance/geography permissions). Organization-level roles are deferred to Bucket 2.
Rationale: The v5 codebase has a fully commented-out Authorization.cs that implemented role×permission checks against the database, plus an active but coarse Position.HasAccess path. The commented code was complex because it interacted with organization and geography trees. With geography gone and organizations flat, a straightforward roles-and-permissions model at the instance level is sufficient and dramatically simpler.
Pattern:
User ──has-many──▶ UserRole (instance-scoped)
Role ──has-many──▶ RolePermission
Permission enum: CanViewLedgers, CanEditTransactions, CanAttestExpenses,
CanManagePayouts, CanManageCrypto, CanAdminOrg,
CanAdminSystem, ...
D8: Crypto chain abstraction
Decision: Introduce an IBlockchainProvider interface abstracting chain-specific operations (address generation, UTXO queries, transaction building, balance checks). Bucket 1 ships with a single implementation for Bitcoin Cash. The HotWallet domain model references a CryptoChain identifier rather than hardcoding BCH.
Rationale: The v5 code has BCH logic woven directly into BitcoinUtility and BackendLoop. Extracting an interface now — while we're rewriting anyway — costs little and prevents a second rewrite when adding another chain.
Alternatives considered:
- Full multi-chain support in Bucket 1: too much scope; each chain has unique UTXO/account models.
- No abstraction: would repeat the current tight coupling.
D9: Exchange rate architecture
Decision: Preserve the BTC-hub model (all rates expressed relative to BTC, fiat-to-fiat derived by division). Replace the current regex-based BitPay parsing with proper JSON deserialization. Add a provider interface to support fallback/alternate rate sources.
Pattern:
IRateProvider
├── BitPayRateProvider (primary — BTC-denominated rates)
├── FallbackRateProvider (secondary — TBD, e.g. CoinGecko)
└── CachedRateProvider (wraps any provider with TTL cache)
ExchangeRateService
├── TakeSnapshot() → tries primary, falls back to secondary
└── GetRate(from, to, date) → lookup from stored snapshots
D10: Backend worker as .NET hosted service
Decision: Replace the v5 BackendLoop console application with an ASP.NET Core IHostedService (or BackgroundService) running in the same process as the API, or optionally as a separate container.
Rationale: .NET 8 hosted services integrate with the DI container, configuration, and health checks. The backend worker needs access to the same repositories and configuration as the API. Running in-process simplifies deployment for Bucket 1; splitting into a separate container is trivial later.
D11: Gitea Actions CI + Docker
Decision: CI runs on Gitea Actions (git.falkvinge.net). Pipeline builds a Docker image containing the .NET backend + pre-built React frontend static assets. The image is the deployment artifact.
Rationale: No existing CI to preserve (CruiseControl.NET is dead). Gitea Actions is workflow-compatible with GitHub Actions. Docker provides reproducible builds and deployment.
Pipeline stages:
dotnet restore+dotnet build+dotnet testnpm ci+npm run build(frontend)docker build(multi-stage: build + runtime)- Push image to registry
D12: Continuation transactions for closed-period operations
Decision: Preserve the v5 continuation transaction pattern. When AddRow is called on a transaction whose date falls in a closed fiscal period, the system creates (or reuses) a continuation transaction dated "now" in the open period, posts the rows there, and links the continuation to the original via FinancialTransactionDependencies. This preserves the v5 principle that financial objects are the working objects and accounting happens transparently.
Rationale: This is not an edge case — it's the normal path for any operation touching a financial object whose original transaction predates the close. Expense claim edits, invoice re-attestation, payout processing against closed-year items all hit this path. Without it, routine operations on historical items would fail.
Future improvement (Bucket 2): The silent redirect is hard to reason about. A better UX pattern (explicit user-visible rebooking) should be explored after Bucket 1 is working.
Storage: Continuation links use FinancialTransactionDependencies with type FinancialTransaction and ForeignId = parent transaction ID. RecalculateTransaction aggregates balances across the chain before computing deltas.
D13: VAT booking at transaction creation time
Decision: VAT booking splits are part of Bucket 1. When an expense claim, inbound invoice, or outbound invoice includes a VAT amount, the system creates separate transaction rows for the net amount (on the budget/income account) and the VAT amount (on AssetsVatInboundUnreported or DebtsVatOutboundUnreported). VAT amounts are entered manually (total + VAT portion in cents), not computed from rates.
Rationale: VAT splits affect the ledger structure from the moment transactions are created. Deferring this would produce ledger entries that differ structurally from v5, breaking migration verification and making future VAT reporting harder to retrofit. European tax reporting requires VAT to be separable from the moment of booking.
VAT-related accounts in the chart:
AssetsVatInboundUnreported/AssetsVatInboundReportedDebtsVatOutboundUnreported/DebtsVatOutboundReported
Per-org configuration: VatEnabled controls whether VAT fields appear in the UI and whether booking splits apply. DefaultVatRatePercent (integer, e.g., 25 for Sweden) is used to pre-fill outbound invoice VAT amounts; it has no effect on inbound documents where VAT is always manual entry. Future Bucket 2+ work can add country-specific rate tables, item/service class rates, and reverse charge automation.
D14: Self-attestation allowed
Decision: v6 allows self-attestation by default. A user who submits an expense claim and is also the budget owner can attest their own claim. This preserves v5 behavior and is essential for sole-proprietor organizations.
Rationale: Strict separation of duties would make the software unusable for the primary use case (sole proprietor doing all accounting). A future per-org toggle for larger organizations can be added in Bucket 2 with organization-level RBAC.
D15: MariaDB/Galera cluster awareness
Decision: The database layer must be compatible with MariaDB running Galera consensus replication. Specifically: LAST_INSERT_ID() MUST be read on the same write connection that performed the insert, never on a read replica. This is the fundamental reason the read/write credential split exists and why write stored procedures return the identity via SELECT LAST_INSERT_ID().
Rationale: In a Galera cluster, a freshly inserted row may not have replicated to read-only nodes when the write returns. The v5 architecture already handles this correctly by design — v6 must preserve this property.
D16: ObjectOptionalData denormalization strategy
Decision: v6 denormalizes the v5 ObjectOptionalData EAV table into first-class columns on the entities they belong to. The migration tool maps each kept EAV key to a concrete v6 column. EAV keys for cut features are not migrated.
Rationale: The EAV pattern was a pragmatic choice in v5 for evolving the schema without DDL migrations on every change. With a clean v6 schema, there's no reason to preserve the indirection — first-class columns are type-safe, indexable, and self-documenting.
Key mappings (kept, denormalized to columns):
- Org:
EconomyEnabled,BooksClosedForYear,Currency,FirstFiscalYear,VatEnabled,VatReportFrequencyMonths,TaxAccount,TaxOcr,GovtRegistrationId,TaxAuthority,FinancialMailName,FinancialMailAddress,LogoLandscape,LogoSquare - FinancialAccount:
NativeCurrency,BitcoinPublicAddress,BitcoinPrivateRoot,BitcoinPublicRoot,BankTransactionTag,EnabledForExpensing,EnabledForInvoicing,OwnerUserId,AutomationProfileId - InboundInvoice:
Description,ExactBitcoinAmountSatoshis,NativeCurrencyCode,NativeCurrencyAmountCents - OutboundInvoice:
CreditedWithInvoice,CreditsInvoice,ToUserId - FinancialTransaction:
BlockchainHash - FinancialTransactionRow:
BitcoinDerivationPath - User (from Person):
BankName,BankClearing,BankAccount,BitcoinPayoutAddress,PreferredCulture,NeverMail,ResetPasswordTicket
Dropped: All governance, geography, membership, activism, forum, party-era, and PII keys.
D17: Foreign currency amounts on transaction rows
Decision: v6 preserves the NativeCurrencyAmountCents / NativeCurrencyCode tracking on financial transaction rows. When a transaction row is in a currency different from the organization's presentation currency, both the presentation-currency amount (in the main row) and the foreign-currency amount are stored.
Rationale: This is essential for multi-currency accounting. An org booking in SEK that receives BCH deposits needs both the SEK ledger amount and the original BCH amount for audit trail and reconciliation.
D18: Localization strategy
Decision: v6 Bucket 1 ships with English (primary), Swedish (secondary), and German (should-have). Translation packs use standard .NET resource files (.resx) for the backend and a JSON-based i18n system (e.g., react-i18next) for the frontend. AI-assisted translation generates initial packs; human review for Swedish.
Risks / Trade-offs
[Risk] Migration data fidelity — ETL could introduce rounding errors or miss edge cases in financial data, leading to legally incorrect ledger presentation. → Mitigation: Verification pass compares rendered output (balance sheets, P&L, individual transaction details) between v5 and v6 for all closed periods. Migration tool includes checksums on financial totals per organization per year.
[Risk] Stored procedure maintenance burden — Handwritten stored procedures are tedious to maintain and lack tooling support compared to ORM-generated SQL. → Mitigation: This is an accepted trade-off. The security benefit (write credentials can only execute specific procedures) outweighs the maintenance cost. Procedures are version-controlled and tested.
[Risk] SingularFactory reflection — The reflection-based factory for hydrating Logic types from Basic types could be fragile during the rename and type reorganization. → Mitigation: Warm the factory cache at startup; fail fast with clear errors if a Basic type lacks a corresponding Logic type. Add integration tests that verify all Basic→Logic mappings.
[Risk] BitPay API deprecation — Current rate data comes from a single source. If BitPay changes or removes the endpoint, exchange rates stop updating.
→ Mitigation: The IRateProvider abstraction with fallback support (D9) means a secondary provider can be added without code changes to the rate consumption path.
[Risk] Large rename scope — 6,146 occurrences of "Swarmops" across 1,371 files. Mechanical rename could miss edge cases (config files, SQL strings, hardcoded paths). → Mitigation: The rename happens as part of the rewrite, not as a find-and-replace on v5 code. Each subsystem is rewritten under the new namespace, so stale references are caught at compile time. SQL stored procedures don't use the Swarmops name internally.
[Risk] Frontend feature parity — The v5 UI has accumulated subtle behaviors (EasyUI grid customizations, jQuery widget interactions) that may be lost in translation to React. → Mitigation: Spec each frontend page's functional requirements independently of implementation. Use the v5 UI as reference for behavior, not for code.
[Risk] ObjectOptionalData migration surface — v5 stores hundreds of entity metadata fields in a generic EAV table. Missing keys during migration means losing org settings, account metadata, or financial configuration that doesn't live in the main entity tables. → Mitigation: D16 provides an explicit mapping of every kept EAV key to a v6 column. Migration includes an audit step that reports any EAV keys found in v5 data that are not in the mapping (either migrated or explicitly listed as dropped).
[Risk] Continuation transaction chain correctness — The silent redirect of rows from closed-period transactions to continuations is subtle. Recalculation must aggregate across chains correctly, and migration must preserve existing chains.
→ Mitigation: Integration tests specifically for continuation chains: create chain, recalculate, verify balances. Migration preserves FinancialTransactionDependencies with type FinancialTransaction.
[Risk] Galera replication lag — Read-after-write on a different MySQL node can return stale data.
→ Mitigation: D15 ensures LAST_INSERT_ID() is always read on the write connection. All write-then-read patterns use the write credential connection.
Migration Plan
Phase 1: Foundation (no user-visible features)
- Scaffold .NET 8 solution with new project structure (
UsingCash.Common,.Basic,.Database,.Logic,.Api,.Backend,.Migration,.Tests) - Create v6 MySQL schema (DDL + stored procedures for financial core, AP/AR, VAT accounts, crypto, users/orgs)
- Build
DatabaseConnectionfactory with credential split (Galera-aware: LAST_INSERT_ID on write connection) - Port
Money,Currency,BasicFinancialAccount,BasicFinancialTransaction, etc. - Wire up CI pipeline (Gitea Actions + Docker + lint)
Phase 2: Financial core + Migration (gates all later phases)
- Port accounting Logic layer (accounts, transactions, tags, budgets, period closing, continuation transactions, VAT booking splits, foreign currency row amounts, OrganizationSequenceId)
- Build ETL migration tool with ObjectOptionalData denormalization mapping
- Run multi-level verification pass: balance sheets, P&L, per-account-per-year checksums, transaction-level spot checks
- Port exchange rate system with proper BitPay JSON parsing and micro-BTC scaling
Phase 3: API + Frontend shell
- Implement REST API endpoints for ledger data (read-only first)
- Scaffold React app with MUI, routing, auth (session cookies), i18n (EN/SV/DE)
- Build ledger viewing pages (chart of accounts, inspect ledgers, balance sheet, P&L)
Phase 4: Write operations
- API endpoints for expense claims, invoices (with VAT booking), payouts
- Frontend pages for AP/AR workflows
- Attestation (with self-attestation support) and receipt validation flows
Phase 5: Remaining Bucket 1
- Crypto treasury (BCH hot wallet, blockchain ingestion)
- Dashboard and dashboard tasks
- End-of-month workflow (upload + manual review; no reconciliation until Bucket 2)
- Comms/mail, event bus, backend worker (exchange rates, forex P&L, automated payouts, heartbeat)
- User management and RBAC
- Dev impersonation
- Database seeding / bootstrap (first admin user + org on fresh deployment)
Rollback strategy
The v5 database is never modified. If v6 deployment fails, the v5 system (once buildable again, or from the last known working state) can be restored against the original database. The v6 database can be dropped and re-migrated.
Open Questions
-
MySQL/MariaDB version requirement for v6? — The v5 schema uses features available in MySQL 5.7+ / MariaDB 10.3+. Should v6 target MySQL 8.0+ / MariaDB 10.6+ to leverage CTEs, window functions, and improved JSON support? (Must remain Galera-compatible.)
-
Frontend state management — React Query (TanStack Query) for server state, or a more traditional Redux/Zustand setup? Given the accounting domain's read-heavy nature, React Query's cache-first approach seems natural. (Resolve before frontend implementation begins.)
Resolved Questions
-
Auth token mechanism — Session cookies. Simpler for single-domain SPA, easy to revoke, works naturally with the WebSocket auth model (cookie sent on WS upgrade), and compatible with Galera (session store can use the write DB or be in-process for single-instance Bucket 1). JWT adds complexity with no benefit at this scale.
-
File storage path mapping — Configure v6 to read from the existing
/var/lib/swarmops/upload/path via a config setting pointing to the legacy storage root. No file copy during migration. Document metadata is migrated with paths preserved. When Bucket 2 adds S3/MinIO, theIDocumentStorageabstraction handles the transition. -
Intermediate currency calculation precision — Keep
double, matching v5.decimalcauses hard-to-debug issues in practice. Rates are inherently fluctuating snapshots; sub-cent precision on conversions is not meaningful. Documented as deliberate. -
Fiscal year start month — Removed from v6 Bucket 1. v5 assumes calendar-year fiscal years. Configurable fiscal year start is a Bucket 2+ concern at best.