SR Technical Operations Engineer (Validators & Rollups)
Description
We at Pearster are looking for a Technical Operations Engineer (Validators & Rollups) to join an American prominent Blockchain company through our team.
This company is a cloud-based infrastructure provider that powers the global blockchain ecosystem. Its mission is to be the indispensable utility that enables companies and innovators worldwide to build next-generation, Web3-enabled businesses and applications using blockchain technology.
The Role
Keep validators online and un‑slashable. We run production validators across L1/L2s and a focused set of rollups. You’ll run K8s at scale, ship IaC, lead incidents, and write the tooling that makes production boring—while guarding keys, latencies, and SLOs like they matter (because they do).
Requirements
- Location: Asia (remote-based).
- Linux + Kubernetes: Debug real production—networking, storage, rollouts, performance.
- IaC (Terraform/Helm/Ansible): Ship repeatable infrastructure and zero‑touch deploys.
- Go or Python (plus Bash): Automate noise away and build small, sharp tools/CLIs.
- Validator operations: Operate/tune validators and sentry topologies; avoid slashing; manage peers; design backups/snapshots/state‑sync; handle client diversity and version skew.
- Key management & signing: Vault/HSM/KMS; Web3Signer/TMKMS/Horcrux; slashing-protection DBs; rotation and break‑glass procedures.
- Observability that matters: SLOs/error budgets; Prometheus/Grafana; alerts that only page when users are affected.
- Networking & systems depth: DNS/TLS/LB, P2P specifics (ports/NAT/conn limits), kernel/IO tuning, capacity & cost modeling.
- Rollups (targeted scope): Deploy/operate OP Stack/Arbitrum/Polygon CDK components (sequencer/batcher/indexer), wire DA endpoints (Celestia/Avail/EigenDA/AnyTrust), upgrade safely.
Responsibilities
- Run validators: Deploy, upgrade, and tune EVM/Cosmos/Solana validators; manage client diversity; integrate MEV‑Boost/relays; keep latency & attestation/commit rates high.
- Protect keys & stake: Implement secure remote signing, slashing protection, and audited change windows; test restores and failovers regularly.
- Automate everything: IaC modules, golden images, CI/CD, snapshot pipelines, state‑sync validation, replay/load harnesses.
- Operate K8s at scale: Safe rollouts, HPA tuning, storage choices that won’t bite you; multi‑region capacity and budgets.
- Lead incidents (SEV0–2): Isolate fast, roll forward/back, publish crisp RCAs, and ship the change that prevents a replay.
- Limited rollups support: Stand up/upgrade sequencers & batchers, wire DA, manage contracts and params, and keep L1/L2 bridges healthy.
- Build signal, not noise: SLOs/error budgets, useful dashboards, alerts that only page when users are affected.
- Code where it counts: Write/extend tools; occasionally patch client bugs that affect production and upstream when needed.
Why This Role Stands Out
- Impact: Your work lights up production; chain launches, reliability wins, performance gains.
- Pace > Ceremony: Tight reviews, practical standards, minimal meetings, async‑first.
- Growth: Own big surfaces; learn protocol internals while leveling up distributed systems chops.
- Remote‑first: Follow‑the‑sun coverage; humane on‑call.
The Bar: Signals We Care About
- Production validator ownership: Ethereum/EVM, Cosmos/CometBFT, or Solana in production.
- Key security in practice: Vault/HSM/KMS, rotation/backup/restore tested; principle of least privilege enforced.
- SLO thinking with results: Measurable improvements on alert noise, latency, attestation/commit rates, missed slots, MTTR.
- Tooling & code impact: Go/Python utilities, IaC modules.
- K8s & systems trade‑offs: Requests/limits/QoS, storage & kernel tuning, P2P connection limits, canaries/rollbacks.
- Targeted rollups experience: Shipped/operated OP Stack/Arbitrum/Polygon CDK components and integrated DA endpoints safely.
Benefits
- Fully remote work arrangement as a contractor.
- Competitive salary in USD.
- PTO days per year.
- 100% company-covered international certifications.
- Access to coworking spaces worlwide.
- English classes.
- Engaging team-building activities.
- Personalized gifts.
- Welcome kit.
- Referral programs and much more!
- Locations
- United States
- Remote status
- Fully Remote