Robin Cannon, Executive Director and Head of Design System at J.P. Morgan
Transcript:
Chris Strahl:
Hi, and welcome to the Design Systems Podcast. This podcast is about the place where design and development overlap. We talk with experts to get their point of view about trends in design, code, and how it relates to the world around us. As always, this podcast is brought to you by Knapsack. Check us out at knapsack.cloud. If you want to get in touch with the show, ask some questions, or generally tell us what you think. Go ahead and tweet us at TheDSPod. We'd love to hear from you.
Hey everybody, this is Chris Strahl with the Design Systems Podcast. I'm here today with Robin Cannon, back with Robin Cannon. Robin, thanks for coming on the show again. I think we're due a little life update. Tell us what you've been up to.
Robin Cannon:
Yeah. What is it? It's about three years since we last spoke on the podcast. But yeah. I'm now six months into a new role as Head of Design Systems at JPMorgan. After the best part of a decade, I guess at IBM, obviously most of that was all around the inception and growth and leadership of Carbon. Then about a year and a half in IBM consulting, really building a design systems offering on the consulting side so that we're working with clients and partners on how to build, refine, expand their design systems.
Yeah. That's, in large part, what I'm doing at JPMorgan is taking the design system and all my lessons over the last decade or so to really expanding our approach with the Salt Design System, which is open source. You've seen some early releases of that at saltdesignsystem.com, and that's an exciting direction we're going. I'm really enjoying it so far. Six months in. I am as enthusiastic if not more enthusiastic than when I started, which feels like a pretty good sign to me.
Chris Strahl:
That's awesome. I think it's an interesting choice for financial institution. You don't see a lot of open source software coming out of big banking institutions. I'm curious about the choice to make it open source.
Robin Cannon:
Yeah. I mean I think that it's an interesting one, because it's easy to get overly focused on the aspect of it being open source. It's open source, primarily, because there's no exceptionally good reason for it not to be open source. It's still primarily, the design system for JPMorgan and for internal products and applications at the corporate investment bank.
But it's also something that partners of JPMorgan may be able to use if they're, for example, using some underlying backend payment processing, something like that. They have a relationship with us there and they also want to build a consumer facing portal, then here's an option that makes it easier for them to do that. It also demonstrates that JPMorgan and Digital Experience design is a big part of JPMorgan's initiative and drive to improve user experience.
This is a demonstration of we are trying to lead in that field. We are trying to be subject matter experts in the design and frontend development space with a focus on the financial industry. That's quite exciting for me to be able to look at things from a different perspective, particularly having spent so long in the tech specific industry to be doing a design system and talking about a design system publicly in a different industry space.
Chris Strahl:
I love it, personally. I think that as it's been touted all around the industry for years now, adoption is the biggest indicator of success of a design system and any barriers needing to be provisioned in account or have your SSO ID or whatever to actually consume the design system is a barrier. Now, it's oftentimes a low barrier, but it is a barrier nonetheless. Being able to have something that's truly open, I think more organizations would benefit from doing that, even if they only intend to ever really use the design system for themselves.
The mere idea that anybody can use it, consume it, look at it, touch it, and maybe even contribute to it if they want.
Robin Cannon:
For sure. Yeah. I think that Phil Gilbert, who was the general manager of IBM design, I remember having a conversation, because he used to be my boss when I was director of Carbon, that adoption is not the metric that you want to pursue. Enthusiastic adoption is the metric, because you can force adoption through mandates and so on. But what you actually want is to create an environment and an ecosystem where people are excited to use the design system, that they are trusting and confident that it's going to help them do their work and that it's accountable.
I think the other big part of being open source is not just the work and being able to see it and touch it. It's working out in the open as well and saying, "Here is what we're doing now. Here is what we're planning to do next." That people can understand what's coming as well as what's currently here and make decisions based on that.
Chris Strahl:
Exactly. Yeah. The reduction to barriers of people becoming enthusiasts, champions excited about it. I mean, that's how I view that value is by making it something that you don't hoard. The information that exists inside of the design system is valuable to a lot of different people. You can go back and forth all day long about whether it's a strategic asset of the company that should be public or not.
But the reality of it is that by having it public and out there, you can communicate in a very different way. I think that that's really special, especially for a big institution that, like I said, I think that traditionally doesn't necessarily swing that way.
Robin Cannon:
Yeah. I think that fundamentally it also makes you accountable to a broader scope of audience. I don't know. I've been in a Mad Men rewatch recently, and there's a line, Don Draper is talking to some of his writers and he says, "Stop writing for other writers." I think that that's also something that designers can fall into that trap of not designing for the end user, designing for other designers to look at that and say, "That's really cool."
The more we open the aperture of visibility of our work at earlier stages, the more we get a broader view and broader feedback from lots of different stakeholders or interested parties, and the end users get to see things much earlier as well. As a result, we're forced to design and develop for the people who are going to be using this thing. I think that that really keeps us honest.
Chris Strahl:
No. I mean let the light in, man. I'm in. I think the other part of this, too, is you guys have a lot of complexity to a system like this. You're in a situation where you and your team, you're supporting a pretty broad scope of products. There's a lot that JPMorgan does, probably more than I can even envision.
Robin Cannon:
Well, yeah, I'm six months in and there's so much that I still have to learn. It's one of the things that is keeping me the enthusiastic and really fascinated in moving into such a complex industry space and understanding or starting to understand the complexity and the variety of different use cases, different platforms, different situations that will gain benefit from using a design system to build those experiences.
Chris Strahl:
How do you do that with one design system or is it not one design system?
Robin Cannon:
I mean, I think that you can do it with one design system. Certainly, like at most companies, there are more than one design system. There's one more than one design system at JPMorgan Chase for different areas. But I'd like our design system. I'd like the salt design system to expand its footprint to be able to cater for a broader scope of use cases.
We already addressed that in some ways by looking at different densities of experiences. Whether that'd be a lower density, more white space consumer or marketing or brand focused experience all the way up to very high density. I think that's one. I used to joke at IBM that there were applications that their primary interface or the primary user experience was just a data table. I thought that that was true at IBM, but now I'm in a financial institution.
There are lots of situations where that's really true. I think it is a big challenge because there's such a broad scope of use. But I think addressing things around what's an expressive use or an expressive variation, what's a very, very data heavy productive use? Thinking of those user journeys and thinking of those patterns, where do they share characteristics and then where are their deviations?
To what extent do you want to pursue those variants of productive data, heavy use cases and expressive consumer facing use cases? How much of that do you want to capture within single components or within patterns? How much of it do you accept that there is going to be this broader ecosystem where you do empower different lines of business or different channels to extend upon the core DNA of the design system for their own specific needs?
Chris Strahl:
That is interesting. What I'm starting to talk about when I hear things like that is the idea of where do I strategically insert fragmentation into my design system? The systems are all about fighting back against the tidal wave of fragmentation that exists inside of products. I think that we've all decided that maybe there are some products that are worth their own fragmentation, their own system within a system, but there are other products that aren't.
At what point do you make those decisions that basically say, "This is unique enough or different enough than my core foundational system or another product system or another brand system to say this is a moment where fragmentation is actually beneficial to the system as a whole?"
Robin Cannon:
I'd almost approach it the other direction and say, "How much of the DNA can we encourage you to share and evolve from?" Then knowing and acknowledging that that kind of work is going to happen anyway. What can we do to facilitate visibility of all that work and an indexing of all that work so that we can then take action around it? Because it's almost like we or no single entity, no single team is going to be well-placed to say that's unique. That's not unique.
Rather, I'd prefer to encourage an ecosystem where we've defined here are universal principles, here are robust guidelines and opinions on how to build and design and develop good experiences, good applications. Here are maybe some early universal flows and components. But then also here is a structure so that if you want to pursue an extension to a component or a new component or you have a pattern, build that we trust you to have followed all the universal characteristics and guidance, build it, get it into your product and just let us know it exists.
We can surface it. We can provide you with some resources around here is a documentation template for some consistent templating. Here is maybe even a static site template for how to throw up your satellite from the causal design system. We'll surface it. At once we index it, we'll be like, "Oh, there's 15 different versions of this component, or there's 12 different versions of this pattern."
That was already happening. Now we can take some action around it. If three of those versions of that pattern are intentional and the other nine are accidental, well, then let's work to eliminate the accidental ones and let's coalesce with all the relevant stakeholders across different lines of business and channels around the ones that are intentional. That's how we accept that there is always going to be a degree of fragmentation.
But let's make sure it's intentional fragmentation rather than accidental. I think that's the biggest thing is what's intentional and what just happens because there is no common frame of reference or the common DNA.
Chris Strahl:
I mean everybody always talks about gardening and these things. But it sounds like that. You have your tomato plant that has roots and shoots and stems and leaves and buds and flowers and fruit, how you cultivate that, oftentimes it's totally fine for the plant to grow another branch. But every once in a while you want to prune it and because you prune it makes for a healthier, more productive plant.
I think that's interesting where your thought process behind this is rather organic, but with that relatively steady hand towards pruning that I think is interesting.
Robin Cannon:
I think that there's always a point if you don't do that, that the design system becomes a bottleneck. If you try and create too much control and you try and push back too hard against the fragmentation, then you become the blocker for teams. We can't ship our product because you haven't either done or approved this patent that we suggested or this component and therefore we can't progress.
All that means is that you'll hit a point where they'll just start going around the design system and you don't have the visibility. It's a lean into providing some structure and support to things that need to happen anyway.
Chris Strahl:
Yeah. There's a little bit of a snowball there. It's finding the path of least resistance, but also growing as it rolls downhill. You always want the design system to feel like that path of least resistance and by feeling that path of least resistance, you grow those enthusiastic adopters and also the value of the system along the way through contribution.
Is that really the tactic you take and this model where you have contributors to a core system and then you also have contributors to some subsystem that exists at a brand or product level? What does that architecture of systems look like for you?
Robin Cannon:
Yeah. I mean there's two different ways I look at it. But the first one is you have fundamentally that hub and spoke model. Right at the center you have the hub, which is the design system, and then you have the high-level spokes, the line of business, business unit areas.
Chris Strahl:
What would those be for you? Commercial banking or something like that?
Robin Cannon:
Not those kind of things. We might have, for example, payments or markets or something like that as a large line of business. Now there's lots and lots of products and platforms within those line of businesses. But they're more likely to share some patterns or some specific component functionality and therefore they're going to be the ones that are driving that and need to share it at least internally among the teams within that channel.
It doesn't necessarily make sense for us to either try and manage it at the design system level because we don't have that domain expertise. But also, we can't be approving things and they're not necessarily universal. Let's create the opportunity for those spokes to develop and be a catalyst for those lines of business to share among their product teams and have somewhere easy for their product teams to push stuff up.
Chris Strahl:
Right. The idea being that markets data table, to go back to your prior example, might not look anything or behave anything like payments data table because those are very different lines of business.
Robin Cannon:
Yeah. Well, there's two different things. It might not because neither line of business ever really spoke to each other in the past, and that's something we need to address, or it might not because they are very intentional and specific domain differences. Now, we can't scale at a central design system level to be able to capture and cater and manage all that anyway, but we can provide the broader ecosystem and structure necessary.
To move to the other visual references, I think of it more like a funnel of specificity. At the highest level, even above the design system, you'll have the general brand guidelines extending into a design language, and then it becomes more specific at the design system level where you're like, "Here are components. Here is patterns. Here are implementation, execution specifics that are cross-horizontal."
Then at the line of business level, you might have more specifics. Like, "Here are components and patterns that are specific to our domain. Here might be in where we have information on how to hook into the APIs and the data in our area. How do we connect the components?" We don't want to have that at the core design system level and that needs to be more abstract.
Also, that is probably the level of distinction where some of these things are internal, the core design system is open source. Now, we're getting to more business specific and domain specific where these things do need to be internal and maybe there's an internal environment or ecosystem that shares the same DNA, shares a lot of the same look and feel is used and consumed in the same way.
If you're a JPMorgan employee, you would hopefully see potentially a richer catalog of components and patterns than you would if you are just viewing the Salt Design System website and you are not part of the company.
Chris Strahl:
No. That makes sense. I mean there's also an access thing there that does matter at some level, especially if you're talking about how do I wire my componentry up to business logic or to content systems that are proprietary or something like that. The idea being that at some level it stops being about just UI and starts being about here's a holistic experience.
Robin Cannon:
Yeah. Exactly. But we can also reverse it a little bit as well. Our role can also be to surface across the business some of these things that are done at the line of business level or at the product team level. Because I also think that nobody is necessarily the best judge of whether something is reusable. If you are a product team in a particular space building something, and from your perspective it might be very, very specific in its use case.
You're like, "There's no reason why this would be at the design system level. We'll just keep it here." Whereas if we say the default is at least internally, we make it visible. We don't know whether another team in another line of business might suddenly realize, "Oh, actually, that gets us 70% of the way to what we thought what we would need to do anyway, so it saved us a bunch of time." We can be the catalyst for that sharing model.
Chris Strahl:
It's almost like the core foundations part of it. There's obviously direct value there. You're making components and things like that. But a lot of what you're doing is you're providing standards and guidance and then you're creating a platform for discovery of this for other parts of the business that may not be aware of what already exists.
I think that's an interesting way to define the role of a foundational system inside of a big organization and enterprise is there's always of course the support side of it, there's the building side of it, but long term, what it's really about is it's about providing the guidelines for how you contribute and for what is the right way. You even mentioned brand, which I think by the way is really special. I'll come back to that in a second. Then discovery of what other people have built. I think that's a cool way of thinking about the role of those bigger, more central foundational pieces of the systems to systems model.
Robin Cannon:
Yeah. I think so, because I think that however big the design system team gets, the companies and the organizations are so large that you can never scale to provide everything that everybody needs. If you are just saying everything has to come through the design system, what happens if somebody submits something to the design system that's really, really high quality but maybe is very specialized, does the design system team get stuck having to both approve that and then maintain it because it's in the design system without the domain knowledge and without really the capacity to do so?
Whereas if we're providing the building blocks necessary for domain specialists to solve the problems in their domain, and then we are sharing that good work, then suddenly the design system's footprint expands every time anybody anywhere in the company builds something good.
Chris Strahl:
Yeah. No. That's democratization. That's the idea of it is anybody can make a contribution and lead it better than they found it. I love that, philosophically.
Robin Cannon:
People do get nervous about the democratization because they're like, "Well, how do you manage quality?" A little bit of my response is, "Well, you don't." You let the market decide. But if you're creating this index or this catalog, you maybe make completion a little bit analogous to quality and you say, "Provide us these 10 pieces of metadata, yes, no questions." It's like, "Does this have code? Does this have my Figma assets? Has it past accessibility testing so that people can consume it in a fairly informed way?"
But beyond that, you can be like, "Okay. If something clearly is getting lots and lots of interest from lots and lots of teams across the company," then from a design system team perspective, we can go and look at that and refine it. But you broadly let the market decide and refine these pieces of work rather than trying to police all of the quality in the first place, which is when you become that bottleneck and you just can't do it.
Chris Strahl:
This gives me to back to the brand thing. There's this acknowledgement of this higher-level concept of brand standards and everything like that, that exist, that are then represented in the design system. A two-parter, how much do you consider the design system to be a steward of the digital brand? Then back to your quality concept, do you ever end up in a situation where the stuff that is contributed to the design system is at odds with the expression of the brand or is that a part of the standards?
Robin Cannon:
I mean, broadly, we want to be a vehicle for the digital expression of a brand that goes well beyond digital, especially at a company like JPMorgan with such a history and such a multi-platform, multichannel approach to engaging with its customers, many of which aren't digital. The brand expression and the brand realization is far more universal.
But from the digital experience design group within JPMorgan and then from the design system perspective, we're like, "Okay. How does that brand get effectively expressed and what kind of methods and tooling and process can we do to make it easier for teams to use that brand in effective ways?"
Obviously, a lot of that does inform our universal principles and characteristics and guidance. Now that being said, there are different products, different applications, some of which may pursue different approaches to how they present themselves. To some extent, it's outside of the design system team scope to please that to say yes or no.
We can highlight and say, "Hey, you're not approaching this in the way that we would recommend," if it's a particularly wild variance, then you can raise that with the relevant stakeholders and be like, "Hey, this is something that maybe needs to be looked at in a different way. That goes for any company that's not just a JPMorgan thing."
Chris Strahl:
I would love to see a JPM brand expressed as 1970s Art Deco.
Robin Cannon:
Right.
Chris Strahl:
Yeah.
Robin Cannon:
I always talk about theoretically you can create something in the design system that is neon pink if you want to do some overrides and use some tokens and use some theming. But then there's a whole governance issue around that. I think that's my biggest thing. I think the design system should allow and enable technically far more than governance around design and brand expression should allow. You're like, "Don't restrict things from a technical perspective, address that through a broader governance approach to design."
Chris Strahl:
No. I mean you use the G word there, which is scary for a lot of organizations when it comes to the idea of design systems, because lots of folks look at these sorts of systems as a method of control and a very fine grain technical control of implementation. I think that you shouldn't be constrained by the tech inside of your platform. You should be thinking about what is the human service model associated with the design system? When you think about governance, that is innately a process that is about the people, not about the technology enforcing limitations for you.
Robin Cannon:
Yeah. You have to do that. Because if you try and enforce things technically, well then you just make life difficult for yourself when the technology changes. A design system should not be beholden to a particular framework or a particular technology approach.
Chris Strahl:
Because it always changes.
Robin Cannon:
You see. Yeah. Exactly. The Salt Design System is built on React. In 10, 15 years, React is not going to be what people use to build user experiences.
Chris Strahl:
We're all just going to be prompt engineers by then, right?
Robin Cannon:
Right. But whenever the solution is a design system needs to be agile and flexible enough to be able to approach a new technical implementation of the same guidelines and the same processes. The vehicle, the underlying functionality of that vehicle needs to be able to evolve and change. If you've added all your restrictions to behavior into a technological approach, then you just make it more difficult to make those changes.
Chris Strahl:
Yeah. This is actually my biggest reason why I always say that custom tech and design systems is really hard is because you can create a really, really brilliant, elegant design system that works for a very specific point in time and you can get budget for that and you can build that. But then what happens when things shift? Often things shift during the build and then trying to figure out like, "Oh, no. How do I go back to that well and either get refunded to deal with that shift or move or shoehorn in my technology that is going to make that change to whatever's new possible?"
Like you said, in 10, 15 years, it's not going to be React, just like 10, 15 years ago, everything was PHP. I think likewise, you're already watching lots of different JavaScript frameworks and stuff be funny enough asserted again by server-side languages. The pendulum swings in a funny way in technology there.
But the idea of what's next is always something that should be at the forefront of your mind when designing systems like this, because this system needs to have resilience that's going to last for far longer than any one technology will inside of your enterprise.
Robin Cannon:
No. I've been talking to my team recently about when I think about the vision of the Salt Design System. We're still very early in this evolution from our internal design system. We're trying to get to that critical massive componentry so that we've got that parity with the existing design system, a lot of products and applications use, so that migration journey becomes a more realistic endeavor, and we can turn around to teams in JPMorgan and say, "Hey, if you're building something new, Salt is ready and you can trust it."
The next step is to really expand everything around that. It's the guidelines. The why should you use a design system? Here are tutorials. Here's how you get involved. Here's how you contribute. Here are the people involved. I talk about it as once you've got that componentry, you want to build everything else to the extent that if you went back and took away the coded components, it would still have value in terms of guidelines and process and everything else.
Chris Strahl:
There is a lot of value in the kitchen sink of design systems as well. Everything, everywhere, all at once.
Robin Cannon:
Yeah.
Chris Strahl:
But at the same time, any one of those parts should be able to shine individually. It's one of the big things that I railed against for a really long time in the industry was having these systems that are purpose built for a discipline. I think that that's the thing that I don't want to confuse by my last statement is I think that a design system is ineffectual if it just works for engineers or if it just works for product people or if it just works for designers.
There still needs to be that cross-functional value. But those individual pieces of that can be rather small. Docs have value to everybody. A system that lets you organize your assets has value to everybody. What I think is a lot harder is when you say, "Oh, I have this thing that integrates really well into VS Code." That doesn't have a lot of value to design. That's not to say it's not a valuable tool, but it's not a tool that is going to reinforce a systems concept throughout an organization.
Robin Cannon:
Robin Cannon"No. I mean it can facilitate adoption and I think the tooling aspect is really beneficial for addressing some of those pain points and those obstacles and frictions for adoption, whether those are designers and developers just having good tooling. I always think of the design system is the place you go first. Then if you have a discipline-specific need, then you can branch off and go to the discipline-specific areas of that design system.
But the design system is where you have the shared language and the design system is where you have the conceptual integrity. The reference like, "Here is the concept of how we are building experiences." That's relevant and useful to designers, developers, or product owners, and product managers.
Then you can go and take the assets and use the code or get the design files or whatever you need for the next steps to actually implement. But the design system is more than just an implementation tool. It is that broader guidance and the integrity of the vision.
Chris Strahl:
Yeah. I love the idea of the design system representing integrity. That's really cool. Yeah. We use a lot of similar words like trust and all those sorts of things. But that idea that this is where the reason for being lives, I think that's really cool. But I mean, since we're all going to be replaced by AI soon, I wanted to jam on that in particular since the company you work for has very famously rejected other LLMs in favor of building their own.
There's a lot of things going on in this space and it's changing things really, really rapidly. I think that design systems are no exception to this. Given your experience at an organization that's investing very heavily in AI and in a place where I think AI is going to change a lot of the world, I would love to hear about what you think this is going to mean for design systems.
Robin Cannon:
It's interesting. Actually, I did an experiment recently where I had a conversation with ChatGPT with design systems and basically challenging it to do some various things and got very excited very quickly and then very, very quickly disappointed subsequently within the same conversation. It was really interesting. I was like, "Tell me what are some of the aspects of a design system? How would you build a particular type of ... would you build a form using? Does it a design system? How would you check accessibility even? How would you run tests for that accessibility with a language like Jest or a framework like Jest?"
It spat out some fairly coherent answers. Then I got a little bit more specific. I know Carbon from having worked on Carbon so long, and obviously there's a certain degree of fairly limited subset right now of Salt Design System components that are publicly available as well.
I asked, "How would you build a credit card submission form using Carbon?" It spat out a bunch of code for a form that used the right class names and stuff, and it was coherent. Then I asked it, "Okay. Well, how would you do that with Salt?" Because still a subset of components. It just made it up. Literally, just made it up.
I started delving into things on that basis, and you've probably seen the reference to AI hallucinations. I came to the conclusion that it's more like ChatGPT will not tell you that it does not know something. AI will not and does not want to admit that it doesn't know something. The next step, I was like, "Okay." I read the Guardian Newspaper online quite a lot.
I was like, I asked it. I said, "If you were using the Carbon design system, what components would you use to rebuild the Guardian homepage?" It listed these components. I was like, "Well, you've listed eight components, only one of which actually exists." I said, "But that component doesn't exist." ChatGPT's response was, "Oh, I'm sorry, I meant this component." It's like, "Well, that component doesn't exist either." It's like, "Oh, no. I'm sorry. I meant this component."
Then you're like, "Oh, now when you talk about AI being a child that needs to be trained," that you're like ... it's just like, it won't say, I don't know, or I'm wrong. I'd be like, "Oh, no. I must have meant this." It's continually making stuff up, even though it should be able to publicly reference this information, whether it be Carbon, whether it be Salt. I even asked it to list out, "Here, list the Carbon components," which you should be able to do fairly easily. It got about 80% right. I thought, "Maybe it's just doing a generic list."
I said, "List the Polaris components." Again, it listed a different list of components that was still only about 80% right. What I find really interesting and potentially worrying about this is I was asking about stuff I knew. I could look at Carbon or Salt or Polaris, and I can see if it's wrong. If I don't have that frame of reference, it looks very, very convincing.
That's where I think we start potentially getting into trouble, because you mentioned it earlier, and I talk about it a lot as well, about trust. How can you trust this stuff right now if you know that it's going to make stuff up or hallucinate or whatever term you want to use, but that it looks convincing? How do we engender trust? Because there are options.
I think that the concept of being able to code in plain English, fundamentally, and tell an AI, "I would like to build an interface that has a navigation menu that has six header items that are this and a profile avatar and a login button and have it spit that out using the design system." That's really exciting. But you have to trust that this stuff is right and good.
Chris Strahl:
Or that it exists. I mean …
Robin Cannon:
A lot of it exists, at least.
Chris Strahl:
Not the highest bar in the world.
Robin Cannon:
Right.
Chris Strahl:
No. I mean, I think that the reality is that most of these AI models, they speak with a level of confidence that we're maybe not used to as humans, where our intrinsic nature is usually to trust what somebody says. But the AI isn't somebody. They're frankly bullshitting you a lot of the time. But they speak with such confidence like, "They must be right or it must be right."
Robin Cannon:
Well, it's fascinating because I think that one of the most important things that I learned in my, certainly career-wise, but in life in general, and I think that goes for a lot of people, is the importance and value of being able to say, "I don't know." To say that with confidence and not try and pussyfoot around that, but actually to say, "I don't know," and then to take action around that.
Until somebody says, "I don't know," then you don't know that there's a gap. Until AI says confidently or lets us know, "I don't know something," how can we effectively train it?
Chris Strahl:
Yeah. That's a good question. It's got much more deeply philosophical than I was planning. But I love the experiment that you did because I think that being able to play around with what's publicly out there. I guess it's also point in time bound. I'm sure that Carbon and Polaris and all those systems have evolved since the time that the model was snapshotted because it is a brain in a jar.
But in that situation, really testing it with the knowledge that you knew about design systems and how that knowledge constructs user experience, I think that's cool. It's a really interesting experiment. I've talked to very few people that have actually gone through the process of saying, "All right. Here's a design system that's public that I do know a lot about. Now here's a website that I know a lot about. I visit every day. How do I make one with the other?" That's a cool concept.
Robin Cannon:
We talk a lot, and there is a lot of talk about AI and what AI can do. I think that we need to interact with AI a lot more each of us to understand what it can do. I think also there's lots of examples of here's conversations or here's what the AI spat out in terms of you see all these examples of all these magazines that previously were accepting short story submissions, like fiction and paid ones, and they now said, "Well, we can't. We've had to stop accepting submissions because there's just a flood of AI stuff."
It's good enough that it's time-consuming to filter out, but not good enough to publish. Then you start getting into the effort there. We talk a lot about the output, but we actually don't interact that much with AI to gain a first-person perspective of what that output is in different situations. I think the other thing that we don't talk very much about, which is actually the more important thing is how do you educate and teach AI for the things that you need?
Because what JPMorgan is going to need or use AI for is very different than what another company in a different space is going to use AI for. Sure, there might be for something like the design system space, I think there are interesting coding with, through plain English, potentially also telling a design system about the needs you have from a component and seeing what it spits out in terms of a component to add to that system.
You can come from two different directions and those will be cross-industry, but there's lots of industry specific needs. I remember reading an article when I was working at IBM about IBM Watson, because Watson was very, very high profile early.
Chris Strahl:
Yeah. No. I mean, wasn't that the one that ... or was that Deep Blue that beat the chess player?
Robin Cannon:
Deep Blue beat chess player, Watson beat Jeopardy or Watson …
Chris Strahl:
That was it.
Robin Cannon:
But I remember reading an article about Watson from the health perspective. There were some companies who were surprised when they bought this AI, this cognitive computing, that it didn't just do everything they needed right out of the box. They didn't understand the complexity of training necessary. I think that how you educate AIs and you have kids, I have kids, we're raising intelligence, we're raising intelligences.
The complexities that go into that, I think that we need to think a lot more about how you educate and teach something that whatever we're going to state it, whether it's cognitive, whether it's artificial, whatever, we're expecting to be a learning thing. Well, then, we better get much, much better at teaching.
Chris Strahl:
No. That's great. It reminds me of a lot of the very simple, but very profound conversations I have with my five-year-old about like, "Dad, what is a sandwich? Having a very in-depth conversation about what is a sandwich and what isn't a sandwich. I've had similar silly conversations as an adult, but my son is actually really trying to learn, how do I define a sandwich?
Robin Cannon:
It's truly earnest, truly earnest from his point.
Chris Strahl:
Yeah. I mean, it's always a joke about is a taco sandwich, is a hot dog, a sandwich, et cetera. But his whole thing was he was eating a meatball sub and he is like, "This is spaghetti, but it's a sandwich." I was like, "Yeah. It's a spaghetti sandwich." He's like, "Well, that doesn't make any sense because there's spaghetti and there's sandwiches and those are different things," on and on and on and on. You can get where that conversation went.
But we have a similar funny, instinctual cultural part of things that AI has learned from that is us talking about sandwiches or whatever. When an AI gets it wrong, it's because there were millions of data points that made that wrong assumption. How do you definitively tell it's wrong? Who are you at your one data point to tell the AI that they're wrong?
I think that that's one of the difficult problems of this is that when you're constructing these models, especially LLMs, and they have all this data, your opinion or your representation of fact is against what amounts to a tidal wave of information that pushes it in the other direction. Yeah. I was talking to somebody about AI the other day that was involved in the OpenAI project. Their thing was is that we're running out of data, that there's not much more publicly available knowledge about humans that is out there for LLMs to consume.
Even a very, very large data set, all the data that was proprietary for JPMorgan behind the firewall or whatever, it would be less than 1% of a change to the LLMs content. I don't know. I think that this is a really interesting problem because training becomes hard in a case where you're a tiny little drop in a very big pond of data.
Then the second side of it is the constraint models become interesting because if the AI will just flat out lie to you, confidently. How do you know it's respecting to the constraints? It's interesting paradox.
Robin Cannon:
Yeah. If the alternative is like, "Okay. Well, we're going to restrict the data sets, so you have AI for specific use cases," but then you're hampering their potential for one of a better word, by restricting how much information they can know in order to make decisions. Even a company is knowledgeable and comprehensive in its understanding of the financial industry and the markets as JPMorgan, if it fed an AI only its own data, then you have an AI that only tells JPMorgan what JPMorgan wants to know.
Chris Strahl:
Right. No. I mean, I wonder what Don Draper would say about that, right?
Robin Cannon:
Right.
Chris Strahl:
Well, hey, thanks Robin. This has been really fun and enlightening. Always a pleasure chatting with you. Thanks so much for sharing. We'll put a link to the Salt Design System in the show notes so folks can check it out. Yeah. Let's talk again soon. Let's not wait a couple of years this time.
Robin Cannon:
Yeah. Well, I mean, we want to see a lot of things happening with Salt, and you will be able to see a lot of things happening with Salt because public over the rest of this year. I'm really excited about the direction we're going. I'm really excited about digital experience design within JPMorgan and the team that I'm managing and the broader environment. I think there's going to be a lot to talk about.
Chris Strahl:
Sounds great. Well, hey, thanks everybody. That's the show. Really appreciate you all. Have a great day. That's all for today. This has been another episode of the Design Systems Podcast. Thanks for listening. If you have any questions or a topic you'd like to know more about, find us on Twitter at the DSPod. We'd love to hear from you with show ideas, recommendations, questions, or comments. As always, this pod is brought to you by Knapsack. You can check us out @knapsack.cloud. Have a great day.
