In this Enterprise Automation Excellence episode, hosts Dan Twing and Tom O'Rourke discuss "automation myopia" - the problem of defining automation needs too narrowly, leading to suboptimal tool selection. They explore how development teams often focus on solving a narrow set of automation requirements, rather than considering end-to-end processes. The hosts advocate for broader thinking that considers operational requirements, business needs, and regulatory concerns beyond just a project’s technical requirements. They recommend wrapping specialized tools within enterprise-wide orchestration systems to maintain visibility of the complete business process while still leveraging the capabilities of specialized automation tools.
Key Points
Development teams often define automation problems too narrowly, leading to isolated point solutions that can't satisfy broader business requirements

• Data pipeline tools like Airflow and Dagster work very well for their targeted data automation tasks but are less suitable for general automation needs
• Such tools can have limited support for enterprise requirements such as business calendars
• Enterprise orchestration tools provide critical capabilities like SLA management, alerting, business calendar integration, and audit logging
• A hybrid approach using specialized tools wrapped within broader orchestration frameworks offers the best of both worlds
• Central automation teams should establish checkpoints in design reviews to help development teams find solutions that balance 

• Establish an end-to-end process mapping requirement - Before approving any automation project, require teams to map the complete process from initial trigger to final business outcome. This forces consideration of upstream dependencies, downstream impacts, and the actual business value being delivered rather than just focusing on a narrow technical solution. The mapping should identify all handoffs, potential failure points, and dependencies on other systems.
• Implement automation governance checkpoints - Position the automation team as a required checkpoint in the design review process for all automation initiatives, similar to security or privacy reviews. Use these checkpoints to ensure enterprise-wide requirements like operational support, business calendars, SLA monitoring, and regulatory compliance are properly addressed, while ensuring that the development project requirements are all satisfied.
• Create a hybrid architecture strategy - Develop and communicate an automation architecture strategy that allows specialized tools (like data pipelines or RPA) to exist within a broader orchestration framework. This gives development teams the flexibility to use tools they're comfortable with while ensuring visibility across the entire process. Provide clear examples for how to "wrapper" point solutions within enterprise orchestration tools to maintain end-to-end visibility without sacrificing specialized capabilities.
  

EAE Podcast Home: https://em360tech.com/podcast-series/enterprise-automation-excellence
Feedback & Questions: mailto:[email protected]

In this episode, hosts Dan Twing and Tom O'Rourke discuss the relationship between observability and automation in enterprise systems. They explore how observability tools and standards like OpenTelemetrycan improve automation orchestration by providing visibility into the entire business process ecosystem. The hosts note that while observability technology is still in early stages for automation, it represents a significant opportunity to enhance orchestration capabilities, reduce downtime, and provide actionable business insights beyond technical metrics. They emphasize the need for standards development and collaboration between automation and observability teams.

Key Findings

• Observability is defined as "the collection and organization of data from the whole enterprise ecosystem," providing visibility into business processes

• Orchestration and observability are highly linked - effective orchestration requires awareness of the systems being automated

• Current observability solutions are still in early stages for automation and lack standardization

• OpenTelemetry currently lacks standards specifically for automation data

• Automation systems need to both consume observed data and be observable themselves

• Rich data models in automation tools make standardization challenging

Key Takeaways

• Automation teams often troubleshoot problems outside their systems - in applications, middleware, or external integrations

• Automation is becoming "too critical to be left in the dark" as its importance continues to rise

• Automation and observability teams will likely be separate but need to collaborate closely

• The EMA RADAR report for 2025 will include new metrics for observability capabilities

Action Items

1. Initiate a pilot project by first identifying what observability platforms your organization is already using, then selecting a specific automation problem to address using that platform.
2. Advocate for open standards rather than accepting proprietary or in-house solutions. Push for OpenTelemetry standards for automation data to increase interoperability and create end-to-end views of your processes.
3. Build collaboration between automation and observability teams focusing on developing relationships where both teams understand how their systems need to interact — with automation tools both consuming observability data and making their own data observable to other systems.

References

EMA Webinar: Unlocking the Future of Observability: OpenTelemetry's Role in IT Performance and Innovation

In this episode, hosts Dan Twing and Tom O'Rourke discuss the complex relationship between change management and automation. They explore how these two control functions must work together despite their inherent tensions--automation requires stability to function reliably, while modern business environments demand frequent changes. The conversation examines how automation teams navigate changes from multiple sources: application updates, infrastructure changes, automation tool updates, citizen developers, and external vendors. The hosts emphasize the need for balanced governance that enables business agility while maintaining system integrity, noting that emerging technologies like AI will further complicate this balance while potentially offering future solutions.

Key Findings

• Change management and automation are both control functions designed to minimize risk, but they approach this goal differently
• The complexity of today's technical environments has increased faster than the tools to manage changes have improved
• Modern business processes often span multiple organizations and geographical locations, creating more complex change management scenarios
• Smaller changes that don't rise to the level of change committee review can still have significant impacts on automation systems
• The pool of people making changes to automation has expanded beyond dedicated automation teams to include various IT roles and citizen developers
• External changes from cloud providers and SaaS vendors can impact internal systems without advance notice
• Automation systems themselves require change management as they receive updates and patches
  

Recommendations

1. Develop a Change Classification Framework - Create a system to categorize different types of changes (application updates, infrastructure changes, tool updates, etc.) and establish appropriate governance for each category.
2. Implement Robust Monitoring Systems - Deploy monitoring solutions that can detect anomalies in automation performance to quickly identify impacts fromunannounced changes.
3. Establish Knowledge Sharing Protocols - Schedule regular knowledge transfer sessions between automation teams and citizen developers to educate on potential system-wide impacts of local changes.
4. Define Clear Governance Boundaries - Document which types of changes require formal change management review versus those that can be implemented with lighter governance.
5. Implement Version Control for Automation - Apply version control practices to automation definitions to track changes and enable rollbacks when necessary.
6. Create Reusable Automation Components - Develop standardized, reusable automation patterns that can be centrally managed but locally configured to reduce the proliferation of unique automations.
7. Review Vendor Change Notifications - Establish a process to proactively review and assess change notifications from external vendors and cloud providers.
8. Maintain Testing Environments - Set up sandbox environments where even small changes can be tested before being deployed to production systems.
9. Conduct Regular Governance Reviews - Schedule periodic evaluations of your change management practices to ensure they remain effective as automation capabilities expand to more teams.
   

Questions and Comments

[EAE Podcast Home](https://em360tech.com/podcast-series/enterprise-automation-excellence)
Contact Us: mailto:[email protected]

Dan Twing and Tom O'Rourke discuss EMA's latest research report titled "The Future of Workload Automation and Orchestration: Driving Digital Transformation with Orchestration and Generative AI". The hosts explore the evolving landscape of workload automation, highlighting how it has become a central hub for orchestration across enterprises. The conversation covers key trends including growth in the number of jobs, staffing and skill challenges, the rise of observability, and the impact of AI on automation strategies. The research shows workload automation tools are evolving into enterprise-wide orchestration platforms that connect business processes beyond just technical automations.Key Findings

• Continued job growth in workload automation, with 20-25% of organizations seeing about 10% growth and another 30% growing in the 10-25% range, though 16-17% of organizations now report staying at the same level
• Lack of skilled resources has risen to become the #1 or #2 challenge for implementing automation, even surpassing security and compliance concerns
• 75-85% of organizations are now focusing specifically on orchestration as a strategic objective
• Organizational structures are becoming more diverse, moving from centralized teams to a mix of centralized oversight with decentralized teams and specialized groups
• Observability has become increasingly important as automation expands to business outcome processes
• 97% of organizations expect AI to significantly impact their workload automation strategy within 2-3 years, with 18% saying it already has

Recommendations

• Build a comprehensive automation skills development strategy
• Develop a structured approach to address the critical skills gap in workload automation and orchestration. This should include creating formal training programs, establishing mentorship opportunities, and potentially partnering with vendors and professional services organizations for specialized training.
• Integrate observability into your automation strategy
• Invest in robust observability capabilities that provide visibility across all automated processes and systems. Implement monitoring solutions that collect telemetry data in standardized formats (such as OpenTelemetry) and integrate this data with your orchestration platform.
• Develop an AI adoption roadmap for automation_Create a strategic roadmap for incorporating AI into your automation initiatives over the next 12-24 months. Start by identifying specific use cases where AI can provide immediate value, such as anomaly detection, predictive analytics for job performance, or intelligent workflow routing. Allocate resources for AI experimentation and establish metrics to measure the business impact of AI-enhanced automation.

Show Links:Enterprise Management AssociatesEMA Future of Workload Automation and Orchestration ReportEMA webinarContact Us: mailto:[email protected]

This episode discusses the role and evolution of Automation Centers of Excellence (CoE) in enterprises. The hosts, Dan Twing and Tom O'Rourke, explore how CoEs have become critical organizational structures for managing automation

 Key Points

-CoEs are an organizational pattern found in large enterprises to coordinate important cross-group initiatives by streamlining the sharing of technical and organizational knowledge

-The majority of automation organizations have either formal or informal CoEs

-CoEs' roles vary from controlling all automation changes to consultative support of citizen developers

-Automation CoEs are expanding, with 84% reporting moderate to significant growth

-Automation CoEs can become a bottleneck if they are under-resourced or their work is deprioritized 

-CoEs can provide essential governance and risk management function, assisting citizen developers in complying with business policies and technology standards

-Looking beyond automation, CoEs will be crucial for integrating new technologies like AI with existing systems

Dan and Tom discuss Tom's observations and learnings from the 2024 Gartner IT Infrastructure, Operations & Cloud Strategies conference in Las Vegas. 

- **Generative AI**: Gartner believes that we are still in the early days with respect to AI, with many immediate opportunities for productivity improvements that need to be balanced with potential risks and unpredictable costs, highlighting the lack of best practices and challenges moving from a POC to production.

- **I&O Agile Adoption**: Highlighting enterprises that have introduced Infrastructure Platform Engineering (IPE) and Site Reliability Engineering(SRE), Gartner recommends restructuring the infrastructure organization around these disciplines, looking holistically at the infrastructure as a managed platform, and adopting product management thinking as a framework for managing the evolution of the environment.

Dan has returned from Italy where he participated in an HCL Automation customer event focused on orchestration of business processes as an integrated element of a centralized automation solution.

We're most of the way through 2024 and it's time to start thinking about plans for next year. In many cases, the budget has already been set, but it's still important to think through what your priorities will be and how to allocate time for yourself and your team.

* Data Analytics – more production applications, growing importance in business decision making

* Cybersecurity – continued focus on keeping automation software updated, access management controls

Chris Steffen joins the EAE podcast to discuss how

Enterprise automation is expected to orchestrate critical

Key Ideas

- Automation systems carry high risk due to their

- Cloud and SaaS foundations still require

- Business-critical automation systems often

- Risk assessment is the first step to address

- Limiting access privileges and eliminating

- Changes to systems can impact availability and

- Security teams and automation teams share the

Takeaways for Automation Leaders

- Regularly assess risks from human error,

- Build relationships with security teams to

- Audit access management to identify and limit

- Review change processes for automations, software, and infrastructure to identify mitigations for significant risks.

Show Links

- Chris Steffen

Building on the learnings from a recent EMA survey on mainframe modernization initiatives, we discuss where these initiatives fit into enterprise activities and what automation teams should consider when involved in these efforts.

Takeaways for Automation Leaders