Lead Software Engineer

About This Role Wells Fargo is seeking a Lead Software Engineer.

In This Role, You Will

• Lead complex technology initiatives including those that are companywide with broad impact
• Act as a key participant in developing standards and companywide best practices for engineering complex and large scale technology solutions for technology engineering disciplines
• Design, code, test, debug, and document for projects and programs
• Review and analyze complex, large-scale technology solutions for tactical and strategic business objectives, enterprise technological environment, and technical challenges that require in-depth evaluation of multiple factors, including intangibles or unprecedented technical factors
• Make decisions in developing standard and companywide best practices for engineering and technology solutions requiring understanding of industry best practices and new technologies, influencing and leading technology team to meet deliverables and drive new initiatives
• Collaborate and consult with key technical experts, senior technology team, and external industry groups to resolve complex technical issues and achieve goals
• Lead projects, teams, or serve as a peer mentor

**Required Qualifications:**

• 5+ years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education

Desired Qualification:

• Experience in Software Engineering, SRE, DevOps, or Platform Engineering.
• Strong proficiency in Python for automation and tooling.
• Hands-on experience with Grafana, Prometheus, and Splunk in production environments.
• Solid understanding of SLIs, SLOs, dashboards, alerting, and observability best practices.
• Experience applying AI/ML concepts to monitoring, alerting, or operational analytics.
• Strong knowledge of Linux, networking, and distributed systems.
• Experience with Cloud platforms and Kubernetes/OpenShift.
• Proven experience leading incidents, RCAs, and reliability initiatives
• Experience building custom Prometheus exporters or advanced Grafana dashboards.
• Strong Splunk expertise (search, dashboards, alerts, log pipelines).
• Experience operationalizing ML models for observability (AIOps).
• Familiarity with CI/CD, Terraform, Ansible, and enterprise automation platforms.
• Experience supporting large-scale, regulated, or globally distributed systems.
• Improved reliability and performance against defined SLOs.
• Reduced alert noise and faster detection and recovery of incidents.
• Increased automation and self-healing adoption using Python and observability signals.
• Strong observability maturity across platforms and applications.
• Improved MTTD and MTTR through effective use of Grafana, Prometheus, and Splunk.

Job Expectation: Reliability & Availability Engineering

• Own and improve availability, performance, scalability, and resilience of production systems.
• Define, monitor, and manage SLIs/SLOs and error budgets to guide reliability investments.
• Lead capacity planning, performance testing, failover readiness, and disaster-recovery design.

Observability & Monitoring (Grafana / Prometheus / Splunk)

• Design and operate a comprehensive observability stack using:

• Prometheus for metrics collection and alerting
• Grafana for dashboards, visualization, and SLO tracking

• Splunk for log aggregation, troubleshooting, and incident forensics

• Build and maintain golden dashboards and actionable alerts aligned to business impact.
• Reduce alert fatigue through signal-based monitoring and correlation of metrics, logs, and traces.
• Partner with application teams to define instrumentation standards for metrics and logging.
• Use observability data to improve MTTD, MTTR, and reliability outcomes.

Automation & Python Engineering

• Develop Python-based automation for monitoring, alert remediation, deployments, scaling, and recovery.
• Build self-healing workflows integrated with Prometheus alerts and Splunk signals.
• Create reusable automation frameworks and internal SRE tooling.
• Embed automation into CI/CD pipelines to improve deployment safety and reliability.

AI/ML-Driven Reliability (AIOps)

• Apply AI/ML techniques to observability and operations use cases, including:

• Anomaly detection on Prometheus metrics
• Log pattern analysis and correlation in Splunk
• Predictive capacity and trend forecasting
• Noise reduction and intelligent…