The Mission: Simplifying Mainframe Security
This summer, I had the incredible opportunity to work at the Lisle office as a Next-Gen Mainframer in UX Research. Building on an academic foundation in Cognitive Science focusing on Psychology/HCI and AI from UT Dallas and prior UX research experiences in distributed environments, I was eager to dive into the unique challenges and immense potential of the mainframe world. From day one, the warm welcome and collaborative spirit of the Broadcom team, including UX colleagues, set the stage for an immersive and impactful twelve-week journey.
The mission this summer? To contribute to making the mainframe experience intuitive for next-gen mainframers. This was the vision echoed by one of our UX team members I had a 1:1 with during my first weeks in the role: 'make mainframe usable for mom.' This vision reflects the current overall initiative in UX—modernizing the mainframe experience. This is crucial, as the most experienced users will soon reach a well-deserved retirement, and to enable the next generation of mainframers in our customer organizations to seamlessly continue using the products they’re familiar with. Currently, this involves simplifying traditional mainframe interactions and transforming them into a modern, GUI-based, user-friendly experience.
My primary focus was on the Security Insights (SI) platform, a web-based tool designed to help mainframe security professionals and auditors analyze access controls and identify potential security risks. Its core purpose is to empower customers to gather data for compliance reports, a critical function in the face of ever-increasing regulatory and audit demands. The challenge was significant but achievable. Currently, mainframe compliance reporting methods are inefficient, requiring security professionals to recall complex 3270 commands. Many banking customers have resorted to manual workarounds due to the lack of an easy-to-use solution for comparing granular security entitlements.
Mainframe Security Administrators need to quickly replicate user permissions when there are organizational changes, allowing new personnel to perform their roles, such as approving loans. For this error-prone process involving thousands of settings and privileges on the mainframe, auditors need to verify compliance by comparing specific security entitlements to ensure the appropriate permissions are given. As senior mainframe users retire, new mainframers may lack awareness of necessary commands and prefer a more user-friendly solution. The existing process involves auditors requesting direct screenshots of live 3270 command entries, indicating distrust in non-native software reports. Overcoming this trust barrier is key to our solution’s adoption.
Project: Driving UX for Security Insights
Working alongside fellow researchers (Matthew Mills and Ryleigh Quinn), designer (Jim Lofton), and product team (Chip Mason and Thomas Melzer), I led the usability testing effort for the early-stage GUI-based interface designs for 3 key features for SI:
-
Runtime Comparison: Comparing security permissions at an account/user level at run-time.
-
Full Report Delta: A broader report-level comparison to track changes over time.
-
Scheduler: An in-built automation for report scheduling.
Who were we building for? Our primary target persona was the Mainframe Security Professional, tasked with managing granular permissions and fulfilling audit requests by providing security comparison reports to auditors to verify compliance. Understanding their roles and specific pain points was crucial.
Our UX Designer Jim Lofton developed a prototype—a functional representation of the interface—to conduct usability testing with representative end-users. This prototype allowed us to simulate key functionalities, such as visually identifying modified, removed, or added items in a Full Report Comparison, or showing side-by-side attributes in a Runtime Comparison, testing initially for the Top Secret External Security Manager (ESM) environment. Our methodology was rigorous: We conducted usability testing with three internal security Subject Matter Experts (SMEs) in one-hour sessions. To ensure our research truly delivered actionable insights, I followed a two-step analysis process:
-
Initial Debriefs: Immediately after each session, I held quick discussions with our research team (and designer Jim) to capture fresh observations and spot early trends.
-
Affinity Mapping Workshop: Then, I organized an affinity mapping workshop for our UX team to systematically organize the hundreds of individual observations into overarching themes. This collaborative approach built consensus and directly translated raw user feedback into actionable insights.
Uncovering Critical Insights & Tangible Impact
Our testing yielded 13 actionable insights, categorized by severity (5 High, 2 Medium, 6 Low). The direct impact of our work was clear: 38% of these insights were taken forward by the product team, resulting in 4 updated Rally items and, crucially, a saved development effort.
Our most significant high-impact findings included:
-
Prove Report Integrity—Chain of Custody: Auditors expressed significant distrust in non-native GUI reports, preferring live 3270 commands due to concerns about data alteration. This reinforced the critical need for verifiable evidence within SI to build robust audit credibility. In the short term, we have implemented a checksum (a digital fingerprint) as a temporary fix. However, this insight directly validated our architecture team's plans for a backend "provenance solution," a more permanent solution currently on our backlog.
-
Trust the Data Source—Data Freshness: Users need confidence in data freshness, especially given that SI leverages periodic data loads from the CIA database, which are not real-time. Ensuring the "last refreshed" timestamp is prominent and transparent allows us to manage expectations and build trust.
-
Provide the "Why." Not Just the "What": Auditors are looking for "security intelligence"—the context and reasons behind security issues, not just lists of entitlements. This validated our long-term product vision: integrating data from our security products (CEM, Cleanup, and TAMz) into SI to provide a holistic view of the security posture, a goal already on our roadmap.
-
Strategic Insight on the Scheduler: We found that users have a strong reliance on existing external mainframe workload automation tools (like ESP and CA7) for report scheduling. This crucial insight allowed us to strategically put the in-built scheduler feature on hold, saving significant development resources and allowing the team to prioritize the other critical features, Runtime and Delta comparisons, for which design mock-ups have already been updated.
Findings Discussion: I involved design and the product team in a findings discussion after testing for direct comments and alignment. Directly hearing from users about their pain points, like “Need to show auditors data hasn’t been altered” and “Auditors don’t trust non-native software,” and involving the stakeholders from the start resulted in the 4 updated Rally tickets for product implementation.
Beyond the Project: A Broader Learning Experience
My summer at Broadcom was about much more than just a single assigned project. It was a broad learning experience, as I contributed not only to the Security Insights platform but also took the initiative to work on the usability testing for the Watchtower Network product, as well as a broader Security Value Stream customer research initiative to make the most of the experience. I made efforts to collaborate closely with my fellow researchers, refining testing scripts and engaging directly with SMEs and customers on numerous calls. Attending the virtual MTE for SI was a highlight, furthering my product understanding.
I valued the connected company culture, actively participating in team lunches, summer worker interlocks, and new hire roundtables. The mentorship from fellow researchers Matthew Mills and Ryleigh Quinn, as well as designer Jim Lofton, was invaluable. I proactively sought numerous sessions with Jim to clarify my extensive running list of questions about the product and the nuanced mainframe security domain. This, coupled with the constant support from manager Ruchi Saxena and our broader UI/UX team and leadership, reflected Broadcom’s supportive culture as I successfully navigated to complete the mission. Throughout this process, I made an effort to connect with many team members, both internally and outside my core team, documenting their suggestions from each interaction.
The opportunity to work in the niche mainframe space has been uniquely fulfilling. It offered fascinating insights into distinct research considerations and challenges compared to distributed environments, such as participant recruitment. I am grateful for this experience and will undoubtedly take forward the lessons learned, skills acquired, and connections made. Thank you, Broadcom.