How to Increase Experimentation in the DoD with Victor “SALSA” Lopez

[00:00:00] Victor SALSA Lopez: How do we enable Airmen, specifically our Spark cells, to get after innovation themselves? How do we increase the rate of experimentation, so that we can find the diamonds in the rough?

A lot is going on out there. Not everything deserves to make it to production. But if we don't experiment well. We won't be at the cutting edge.

[00:00:34] Bonnie Evangelista: All right, here we are again. I've got another defense maverick in the wild sitting with here talking with me. I'm Bonnie Evangelista with the Chief Digital and Artificial Intelligence Office. I've got Victor Salsa Lopez from the Air Force. Victor or Salsa, can you give me a quick rundown? Who are you?

Where do you work? And what's your expertise?

[00:00:56] Victor SALSA Lopez: Absolutely. Thanks so much, Bonnie, for having me and for the CDO team for all that you all do, I'm excited to hang out with you all for this hour. I am a Reaper pilot in the Air Force who has chosen a very odd path as many, I'm sure of the other Mavericks on your show will say I got about 2000 or so hours, I'm still flying actively in the Reaper community flying my combat missions.

But in 2019 I was asked by the vice chief to help set up an artificial intelligence accelerator, a research and development unit at Massachusetts Institute of Technology under the new Department of the Air Force and MIT Artificial Intelligence Accelerator, where we worked on fundamental research and I focused in on small UAS research, so like quadcopter, little tiny airplanes.

How do they work together and how do they work with humans? And kind of a multi-agent-like human-machine team kind of looking towards the TCA future, right? After about three years or so of doing that, I got asked to go and put all of that hard work into practice. And so now I am at AFWERX working on robotics and artificial intelligence.

Part of the Spark branch Spark Directorate under the ARC branch and so our job is to try and find where all these technologies fit within the operational world and make sure that they're tactically relevant for the warfighter.

[00:02:09] Bonnie Evangelista: Are you still a pilot?

[00:02:11] Victor SALSA Lopez: I am still a pilot, still flying. Yeah, I gotta go in a couple of weeks here.

[00:02:14] Bonnie Evangelista: I feel like that's so unusual in a good way that you're An operational user, but you're also kind of in the middle of this. How do I bring technology to my world? How did that, that's, how did that happen? I don't understand.

[00:02:29] Victor SALSA Lopez: Yeah, I think a lot of luck and was it luck plus preparation equals opportunity I think is the winning formula, right? So,  back in 2019, I was finishing up my graduate degree in systems engineering at Georgia Tech and focusing on the rapid development of military technology. Before that, I had gotten an undergraduate degree in astronautical engineering from the Air Force Academy.

And that kind of two things together, I got lucky in my first operational unit that was very forward-leaning and we did a lot of really fun operational tests in combat with things that were brand new that we hadn't tested before and they would ask us to, to use it. And because of my engineering background, I got to help lead a lot of those efforts as I grew into my career field, which was so much fun.

Because of all of that, my resume was highlighted whenever they were looking for airmen or operational airmen to staff the AI accelerator because we knew that it was going to take. More than just the R& E that the S& T folks, if you will are brilliant in what they do. We wanted to make sure that we were able to connect back down to the tactical end user and to the program office to enable the transition of all the new technology.

And that was so important. And so I and 10 other tactical Tactically and operationally relevant, very young captain types, staff, sergeant types staffed this unit to, to get after it. And part of that was because I was leaving my career field too early. We were making a lot of change, and Vice Chief Wilson had the forethought, to bring in those young folks, but that meant that I had to keep flying otherwise the Air Force would get upset cause we were going to rock the boat a little bit too much.

And so we said, Hey, go figure out how to fly. Thankfully the, the Philadelphia. The Pennsylvania Air National Guard out in Willow Grove has been gracious enough to host me. And so they have been hosting my flying hours for the last four or five years now. And I've gotten the chance to fly, with folks from all over the country now, which has been so much fun.

To stay combat relevant. But also still get to be in this S&T and AI World 

[00:04:25] Bonnie Evangelista: How long have you been called Salsa? Yeah.

[00:04:30] Victor SALSA Lopez: are earned in the Reaper community as in many other communities. They are not something that you call yourself. Someone else has to give them to you. I would not have chosen this name and I did not like it when I initially was told that it was going to be mine.

So back in, oh, it would have been 2017 ish. My unit decided that I was worthy of a call sign. And so you go from being the friggin new guy to a named person in the unit. So I was more than just a number. And it kind of signals acceptance into the unit. And it draws its history back to World War I.

 back in the day, before we had things like Blue Force Tracker and Link 16 that could tell us where everybody was and really good radios that,  we could communicate one or even Merck chat nowadays, right? Where we have chat rooms for wartime back then you didn't know who came home until you came home and you'd go to the bar and you would have a roll call and you would have everyone's name up on the board and you go through everyone's name.

And if they respond that they're present, you know they made it home. And if they didn't respond, you knew to have a drink in their honor. And so from that,  heritage and history of the Air Force, we've continued these traditions throughout. And so today squadrons,  bestow these names.

in a roll call scenario where the entire squadron gets together and they deem you worthy of having a call sign. They kick us all out and they go and decide on it. And they bring you back in and they usually will give you your name first and then explain it to you after. I tried to drink my name off I think 10 times before I realized that everyone was really wanting to call me salsa and they had like,  10 different options up there.

They seemed fixated on the Salsa one, and so after, they finally named me, I sat down somewhat upset and I got pulled aside, and they told me why, and the history, and the story, and the thought of the acronym that Salsa stands for, and the story behind it and the amount of love and care that they put into it, and you realize that, wow, like, okay, one, I should be proud that they put this much time and effort and thought into it, And it also signaled the final acceptance right into the squadron, which is such a big deal.

And it, it hearkens back to our, our one days. So, it's a huge source of pride, for pilots. I know it's weird to the rest of the community but it's, it's something that is now with me forever,

[00:06:45] Bonnie Evangelista: I think the story helps. I'm not gonna lie. I'm much more amenable to calling you salsa or it's not as I say amenable. It's not that it's it's less weird, right? It's custom. It's not my custom, right? But I can dig that. Okay, salsa. Good. Alright, so, pilot turned I don't know, innovator.

[00:07:08] Victor SALSA Lopez: Yeah, robotics, AI, something. Yeah. I don't what my job is, but that's what makes it fun.

[00:07:13] Bonnie Evangelista: So one of the first conversations we ever had I knew you were a defense Maverick because you were telling me a story about how you were using the doden to do experimentation. And I remember I said, you got to, you got to say that to me, say it slower.

And I was like, what are you doing? And, and how is this working and whatnot? So tell me a little bit more about like something you're working on or like, what does life look like for you today? As it's also the innovator

[00:07:37] Victor SALSA Lopez: For sure. So I think part of our mission in AFWERX and specifically within the Spark directorate is how do we enable Airmen specifically our Spark cells to get after innovation themselves. How do we increase the rate of experimentation so that we can find the diamonds in the rough?

A lot is going on out there. Not everything deserves to make it to production. But if we don't experiment well. We won't be at the cutting edge and we have disastrous consequences if we don't. Make it to the cutting edge. And so one of the things that we found when I was at the accelerator was infrastructure for running AI code.

We knew that this was a relatively hard thing. We needed a place where airmen could go and run code. And they, they would tell me that it was very difficult to do that, that,  finding a place and getting the approvals and getting an authority to operate and authority to connect or things that are difficult to do.

And, and in large part, they're, they're right. But one thing that a lot of folks just didn't know about was this DoD information system that you mentioned. 

[00:08:34] Bonnie Evangelista: it was the Dren. I'm sorry. Not the don't know what he's thinking. I'm sorry.

[00:08:38] Victor SALSA Lopez: you're fine. So the DREN, yeah, so, or D REN, I think it depends on who you heard it both D REN or the DREN is part of the High-Performance Computing Modernization Program that no one in the DoD seems to know about, but it is a DoD program owned by DISA and the DoD.

It's been around for over 25 plus years. It's had over 5 billion dollars invested into it with an annual budget of around 270 to 300 million dollars. And on this network, defense engineering network you have a whole bunch of affiliated research centers HPC arcs, they're called.

So, these arcs are little nodes on the network and all these little nodes have what people in the AI world care about. The thing that enables AI is data. Compute and algorithms. We have the algorithms on open source. You can download those on the internet today. Sometimes we have data.

Sometimes it's not well organized, but we can grab that. What we kept missing was the compute part. And so we started looking in the DOD and,  within CDAO  different government cloud solutions. And those have been great, especially when we want to scale, but when it comes to experimentation.

The small stuff, a couple of users just trying to figure out if this thing works. I went to the HPC ARC, specifically one up in Rome, New York, and the folks out in Rome, New York at AFRLRI, which is the Information Directorate of Air Force Research Laboratory. have an affiliate research center that is connected and has the authority to connect to this defense research and engineering network.

And there they have compute, GPUs, and graphical processing units, the kind of backbone that does all of the hard work in AI. They have places where we can store data. We bought two petabytes of extra data that we wanted to store on there. for the low price of a one-time cost of 100k. We can access the compute nodes, even though it's up in New York, from anywhere in the world over a terminal, which is just like a command line interface, which you might see in the Matrix.

my team uses that plus their CAT cards to log in. And we get immediate access to all this computing. And we're starting to use that now in different ways to start prototyping software and prototyping AI models that will allow us to kind of move the ball forward and go and put those on systems that can make a difference.

And this is something that's already been paid for. So the folks up in Rome, New York, do not charge me any money for using their systems. I burned up a motherboard, my team did, a couple of weeks ago. It was completely fried. Because we pushed it a little bit too hard, or maybe it was old, we're not sure.

But they didn't, they weren't upset about it. They just knew, hey, this is part of the thing. They moved us to a new computer, all our data was still there, and we continued working. And I started to now help seed a lot of our Spark cells, and a lot of the folks who need that capability. To have computed, to be able to put it on a place that's IL5,  impact level five certified for controlled and classified information for CUI data.

And they're excited about that opportunity. And I'm just so thankful that this organization exists.

[00:11:28] Bonnie Evangelista: So what kinds of things are, or capabilities or tools or models are you playing with in this environment?

[00:11:36] Victor SALSA Lopez: Man, so,  we have a couple of programs within AFWERX within the Prime Directorate. We have some,  autonomous vehicles. So we have Autonomy Prime there. So we're looking at how do we enable like groups one through three. So these are quadcopter-sized to kind of like your VBAT-sized maybe six-foot tall airplanes.

How do you enable those to be autonomous? How do you get them to have an autopilot? But also how do they start to understand their world? So a lot of people will talk about autonomy in the autonomy space. I will talk about it from a robotics perspective, which is a bit different. Within the robotics space, when you're trying to build a robot, your robot needs to have a few things.

It needs to be able to interact with the world, somehow. It needs to be able to perceive the world. And it needs to be able to make decisions, right? And when you kind of break it down there, you start to understand how to start building and designing your robot in a way that makes sense.

But I also had a lieutenant the other day reach out because he wanted to play with large language models. That was something he was excited about. And I gave him a little bit of information, a little bit more, and he kept doing great things. Lieutenant Nick Brooks, if he's out there.

And Lieutenant Brooks goes out, and I give him some GPUs once I realize that his technical capability is pretty high. And he goes and he knocks my socks off and he gets some demos that I had tried to build, but didn't quite have the time to up to something ready to go and experiment with.

And now he had a graphical user interface for anybody to go and interact with and have large language models on the Defense Research Engineering Network, which is huge. 

[00:13:04] Bonnie Evangelista: you're saying he built that, he built the. 

[00:13:06] Victor SALSA Lopez: Yeah, he took open-source code from the world and went and put it on this network. And realize that, hey, no kidding, you can run this on this thing. And Salsa figured out how to get me access through the HPC ARC in Rome, New York. With AFRLRI to all this stuff. And I didn't do much.

I've connected with him and helped him move things forward. But all of a sudden now he has this incredible capability that he didn't know existed and he had to pay 0 to get it, which is just huge.

[00:13:31] Bonnie Evangelista: yeah, there are so many things on my mind. I'm landing on this idea that what you're the scenario you just described was Lieutenant wanted to play. You gave him the sandbox and. Access to the sandbox and it had those three things you described earlier and he could play. I think that's something a lot of people are trying to do.

There's, I can tell you at the OSD level, Deputy Secretary Hicks released her. I adopted strategy I think it was a couple of months ago now and experimentation is a big element of that. I guess I'm wondering how do we, how do we meet in the middle essentially in terms of. Getting to actual experimentation.

Cause I think when, when you hear someone at the OSD level saying, I want to experiment, there's a lot of churn to figure out how to do that or how to practically implement that. And I think that's where this conversation can be cool in my opinion, because there are a lot of people in between you and OSD.

So how do we meet in the middle, and proliferate more of what you're talking about, from your perspective? Oh,

[00:14:49] Victor SALSA Lopez: experimentation. I think the deputy secretary,  hit the nail on the head there is, is the magic sauce. And it is for two reasons, right? The first is you won't know what works and what doesn't unless you experiment. But the second is we sometimes forget our most valuable resource, right, which is our people.

And while this particular lieutenant is great, and I'm super excited that he's running through all of this he has a day job that he has to get done, and he has responsibility there. But at the same time, if I don't invest in him, From an education perspective, from a letting him fail perspective, and from pointing him in a direction that makes him excited, I'm going to lose the ability to grow him technically into a leader that I'm going to need in the future.

And so, from a leadership perspective, right, how do we meet in the middle? It's really about enabling that experimentation and not directing the direction that it goes. I didn't tell this particular lieutenant how to put something on there. I didn't direct him on which type of model to use. I didn't direct him on what open source thing.

I let him go and run in the direction he thought was interesting. And in doing so, I'm helping build him. But all I've done as a leader is open up the door and let folks walk through it. And so when we talk about enabling experimentation, specifically in the eye domain, we have to look at how we enable those tools and accesses, and if people aren't using them or we don't have good utilization, whose problem is it?

And a lot of times we like to force innovation and the use of tools or make it through an AFI. Or directive. That's usually problematic. Yeah. And that's usually problematic because we don't like to think of ourselves in leadership space as providers of a service, and we don't like to think of our subordinates as customers.

But the day, as a leader, that's really who you are. I am not a person, while I can give an order, those times should be far and few between, unless again, there's an absolute mission that we have to get done tomorrow, right? Then Navi. But in the innovation space, we have to flip a switch and realize that we are the ones providing the service.

And if our customer, which in this case might be a Lieutenant, could be a senior airman, could be an A1C or a private, right? They are our customer. And if they're not using our tools. It is on us to determine why our tool is failing, and that is a personal failure, a leadership failure on our part. And that can, I think, be a difficult thing to.

Sometimes the stand is a leader who is used to being able to lead and give orders and have people follow. It's much harder to lead a volunteer organization than it is a military organization because the spaghetti stream, of efforts moves everywhere, but in the innovation space, that's exactly what you want.

And that's where we get the goodness of experimentation.

[00:17:43] Bonnie Evangelista: Yeah, that's what you're describing is pretty hard, right? I just want to foot-stomp that, but it can be done. I liked your point about bringing people end users operational or not. To the table and, Giving them recognition.

I think that they, in the sense that yes, like we are here to serve you, if that makes sense. And I think a lot of times. They're used to being told what to do, right? And that, that they're,  they're given their equipment and their capabilities, and it's always somebody else telling them what's good for them and maybe flipping the script to your point, like actually saying, Hey.

How can I help you? And, and that, that starts to open up the aperture for what you're talking, you know when you talk about innovation from a practitioner's lens. That's the way I'm, I'm hearing it at least. sorry, I was just really in the moment there. I liked that. Thank you.

[00:18:37] Victor SALSA Lopez: I can't take credit for it. That's something I think I learned from the Colonel who helped set up the AI Accelerator. In my first interview with Colonel Gordon, I was worried that I was not the right guy. I was like, man, I do some, like, maybe some programming on the side, and I've done some engineering, but like, I have not done a lot of AI.

Like, am I the right guy? Do you want somebody else? And in my first initial interview, that was my going-in strategy. And what he told me within the first five seconds of me speaking was salsa, flip the script. What can I do for you? What do you need? And that had such a profound impact on me.

And then while at MIT the university students are very similar. There's an entire month that they give back to the students where there are no academics. The month of January is called the independent activities period where you go and do what you want. Where does your passion lie? And we have to, I think, learn how to balance the military necessity of our job, of getting the job done, flying the combat mission like I have to go do, leaving white space, and giving the opportunity to go and innovate and make something different.

And that, that can be hard.

[00:19:43] Bonnie Evangelista: I want to pick on the infrastructure piece for just a second because I feel like based on my experience, the more critical. Elements to giving room for play. And it has the least conversation, I suppose.

So, I mean, you were just describing you kind of found a workaround, I suppose, in your scenario, but like, how do we think about this more broadly as a collective from your perspective, like can, like, why is it Salsa trying to, end run compute so that.  he can just create that sandbox he's looking for.

[00:20:20] Victor SALSA Lopez: we're really bad at advertising in the DoD. And we don't do it well internally. And, and I am a victim of this or, or I am, I'm guilty maybe is the right word. I don't advertise well either for any things that we have. So Part of it, I think, is just knowing what's out there.  the DAF at least has their big six which is great.

These are data platforms where you can go and put data in there. Cooey. Some of them are, are even at SBIR. Others are at Jay Wicks and you can go and actually. Get on there, put data, run Python code, run SQL R, et cetera, all the nerdy data science things. There are things like the DREN, and there's also an S DREN, right, the secret advanced research engineering network that you can go and access.

That, hey, by the way, has compute, has storage, right? You can go and use these things. And I think it goes back to the user experience as far as The advertisement exposure and ease of utilization. We do a poor job in the DoD at user experience because we don't think of ourselves right as the provider of a service to our customers who might be a senior airman or an A1C or private. For them, what does the user experience look like to log onto your system? How many steps? How many PDFs, how many signatures, how many emails, how many routes? That can be difficult. And if we don't get the UX right, the user experience, right, we're going to continue to fail our airmen because only the ones who have the most to gain, or who are hardheaded enough, like I am, are going to push through all those bureaucratic hurdles to make it happen.

So experimentation isn't only about the infrastructure, but it's also about enabling the infrastructure. And again, that's a bad thing. If no one's using your product. Maybe it's a you thing. What does that experience look like? Maybe it's us. That's the problem. Not necessarily the airmen. And I think if we take that ownership, um.  head-on, we start to understand how to work on what makes it better. I think the trouble for DoD that we don't have may be in private industry in the DoD or the government we don't have a bottom line. If no one uses the product, the contract's still going to get paid. You might just have poor metrics.

So. It's really on us and our amount of leadership and our humility to be able to say that we have failed at advertising these platforms. We have failed at the user experience on these platforms. And we have failed at getting them to the right folks at the right time with the right tactical understanding of where they're going to lead.

[00:22:55] Bonnie Evangelista: Do you have any recommendations for anyone trying to navigate, the spaghetti, as you called it? what can they do? Cause, cause, cause sometimes you're just in a position where that's happening and you need to, you still need to do something anyways, like yourself. So how are you doing it?

What, do you recommend for others?

[00:23:19] Victor SALSA Lopez: I think one of the things that's been massive post-pandemic And it's something that has been written about, right? So General Crystal wrote about this in a book called Team of Teams something that my current boss,  recommended we all read. And now that I was reading it, it made a lot of sense, right?

Though, there are leadership challenges here too that I'll get to. In Team of Teams, General McChrystal talks about the fluidity of some of the Al Qaeda cells that we were fighting and how they would quickly reorganize and continue to maneuver even after maybe we, attacked or were able to eliminate a high-level leader from the organization.

The question was, how do they reorganize so quickly? How are they able to continue fighting? When we keep knocking off their bureaucratic lists. And, the answer was it, it was a team to teams approach. And what I've seen post-pandemic has been interesting. My Microsoft Teams has blown up.

I think I had maybe one team on Teams pre-pandemic, where people tried to post stuff in a wiki. I have lost count of how many notifications I get now with how many groups I have been added to because of this team-of-teams approach. The same thing is happening on other platforms that are both approved and not approved for CUI,  that have public data or CUI data.

 we within AFWERX have supported now Mattermost for the entire DoD. So you can go and get a Mattermost account without having to pay for it. It's already been covered through our contract. That's another place. And that is up at aisle four with an ATO as well as aisle two, depending on your need and needs and use cases.

But this team's a team approach of starting to self-organize and share information and find the folks, right? Through asking questions. Will eventually lead you to the same human. So what we find, at least in the innovation world, is it's the same humans doing the same things over and over again. It's the ones that are hard-headed enough to make the circle.

[00:25:09] Bonnie Evangelista: That's fair.

[00:25:10] Victor SALSA Lopez: And, by the way, they're the same people that are piling up in these chats. Right. And being social about it. So between things like Teams, AFWERX has, has Vision and Ignite as well, which are great platforms to go and post your innovative ideas. So that same lieutenant with the team, now it's a larger team that includes some other folks from AFRLRI, and a group called DarkSaber, which is a ragtag group of volunteers working to build platforms for Airmen and Guardians.

They are,  putting this idea for more computing in Vision. And I think they're like the number one pick. And so for like,  going to get funding, and they want to move forward now in Vision, Oz to fight to get funding for this idea. so what started like one lieutenant playing with a large language model might turn into a no-kidding fully funded project that moves forward for the entirety of D, which is huge.

From a leadership perspective, however, this can be difficult. If you have a mission to get done, your combat mission, how do you allow your airmen, your guardians, your soldiers, your Marine, and your sailors to start self-forming teams outside of your purview? How do you give them the psychological safety to do so?

How do you humble yourself to know that they might be doing something outside of your expertise and that's okay? And allow other people to begin leading your folks. Without your knowledge, and that might seem like hearsay to most of the top level down, right? Like top-down, very bureaucratic, very military structure, but this is something that has happened throughout the ages in warfare, right?

Folks Airdrop into the wrong location and have to form a team. There's a large attack and you're going to form with whoever's around you.  we and we see this time and time again in crisis scenarios. And the military is no stranger to it. The only difference is now we're doing it online and reforming.

And so it's on the subordinates to work to inform their leadership, but also on leadership to give their subordinates the psychological safety to go and do these things.

[00:27:08] Bonnie Evangelista: So on that note you're, you're making me think of a concept that I talked with the AFRL CIO, actually, Alexis Bonnell, and she talks about time as a, as a weapon, because time is the only thing that we share with the adversary. So in this day and age, it's the, we have to figure out how to cheat time, essentially, like we need cheat codes for time.

So, she calls them wormholes. And so where, where can you find like the wormholes and your setting so that you can, and, and, and honestly, I think she's, it's a symbol or an analogy for the bureaucracy. Like, how can we, what are the cheat codes for our bureaucracy so that we can just like, to your point, get a thing done or work to even though like, we're, we can form a team that's.

Maybe not obvious, like nobody works for anybody, but we're just all self-forming and self-organizing like you're talking about so that we can get the mission done. So I kind of want to throw that on you. Like, do you have any good cheat codes? Do you have any good wormholes? Like what, aside from a team of teams, like I got that, but like, is there anything a little bit more granular that you can offer?

And, and that could be,  just like resource-wise, it could be. I'm just, I just have a practitioner's heart, so I'm always like trying to pick on how to do the thing

[00:28:33] Victor SALSA Lopez:  we get this a lot, right? And like one of them, I'll pick on one thing which is contracting, right? Our contracting officers are great.  you, run Tradewinds, which is an amazing tool. But man, like sometimes you need a cheat code, right? It's like, okay, I know I want X or Y thing.

I have a week to get it. I think I found money. How the hell do I buy it? Right. That's, that's hard. One of the cheat codes that not a whole lot of folks use is our SIPR portfolios across the government, by the way, right? So one thing that folks don't know much like Tradewinds, right? Once a company is in the SBIR portfolio.

You can sole source that company as long as it extends that SBIR, any of the SBIRs that they've ever had. They can go from phase one directly to phase three. There's no issue with that, as long as you have the money to go do so. Likewise, the next problem is like, how you find these companies?

Like, how do you know what's in the portfolio? And that's unfortunately,  if you go to appworks. com, I'll plug a little bit here, right? We have this tool called vision where you can go and put your innovative ideas and find innovators from at least the air force across departments of the Air Force.

No issues expanding this across the DOD either. I'm sure if we needed to. And there's also a thing in there called Ignite. And what Ignite is, is a matchmaking tool for companies that have technologies that have been in the CBER portfolio. So that you can go and get your stuff done. So, these are some of those cheat codes that you can use.

Likewise for software, right? You want to run software, but you don't have the like 600, 000 to get your ATO through,  whatever it might be, or your six months or your seven months. You need it faster than that. Can it run on something like the defense research engineering network? Is there an arc near you that can do it right?

HPC. mil is the, you know. The main website and afrl.hpc.mil is on the Air Force side. You can go there and say, Hey, knock on the door. , I asked and they gave me my stuff in a couple of days. Right? So that's a cheat code there for your software. And then cheat codes for advocacy. Not only a team of teams, right?

But the same thing in Vision. You can vote on projects in Vision, and you can join other projects in Vision that match what you want to do. And likely, if you have an idea, the base next to you probably has the same idea, right? Or at least other folks in your career field. But again, how do you self-organize?

You have Microsoft Teams, but we also now have Vision. And you can go and vote with your, and say, like, hey, this is important. And if that makes it high enough, that has visibility to the Chief of Staff level, right? So now you can get into the AppWorks Refinery and refine that idea for no kidding funding and POMs and Stratify right, which allow you to use government and or private industry matching funds to expand the scope of your small business innovation research project which is huge, right?

So that's, that's, those are some of the tools that are out there that allow you to short circuit some of the normal bureaucracy until you build enough advocacy and you've experimented enough that you've had enough success that you've gone through this process. That you're ready, to make this a fully funded, normal, boring business use case.

And you're onto the next amazing thing.

[00:31:42] Bonnie Evangelista: that's awesome. I would be remiss if I didn't add a. A clarification for those out there, cause I am a contracting professional. So when he's talking about SBIR and doing direct awards, it's not a sole source in the technical sense of a sole source. Like you don't need a J and a justification and approval having.

A phase one or phase two sibber or sitter has met the standard of competition. And, but I know like, that's another thing in, in our community, there's different interpretations on how that's executed. Do I have to go through a sole source activity? Like I would a traditional FAR contract I would offer. No, you don't, but the look and feel of it to salsa's point is correct.

You are working directly with a vendor. One-on-one to complete, extend, or derive from their SIPR technology. That's it. No JNA should be required.

[00:32:37] Victor SALSA Lopez: That's why we have you guys to keep us safe

[00:32:40] Bonnie Evangelista: of course, they're so, where do you want to land in terms of how we want to come to a close in our conversation?

What else about experimentation or otherwise resonates with you in terms of leaving us some final thoughts

[00:32:57] Victor SALSA Lopez: for those who have. People in their charge. As we look towards the future, warfare, while nature will continue to stay the same, right? Then, as the character changes, how are you enabling experimentation at your level? And what resources do you need to ask, for the special folks in your unit?

For this, I met a supply troop in Doha at IED last year. Who, before immigrating to the United States was a network engineer and ended up on a special task force because he was a network engineer and they needed that. But his job was supply, not.

[00:33:38] Bonnie Evangelista: is so common.

[00:33:40] Victor SALSA Lopez: Very, right? There's, I've had two linguists, brilliant linguists, who have been amazing Python programmers who've helped me build products very quickly, both under the E5 level. I've had this lieutenant, right? That we talked about today. We had another senior airman become a program manager with us. He was cyber Intel. But he joined the Air Force somewhat to spite his family because he had also gotten into Harvard and MIT, but decided he wanted to enlist instead.

[00:34:15] Bonnie Evangelista: Hmm.

[00:34:16] Victor SALSA Lopez: These are the special folks that our country has decided to give to us. And I think it is on us to figure out how to give them what they need to succeed. Not just within the lens of their career field through a traditional career path, but in the lens of how do we prepare for the next fight? and let them experiment so that we build them both personally and build the department.

[00:34:39] Bonnie Evangelista: All right. Well, thank you. Salsa. Thanks for giving me a little bit of your time. Creating some space for some cool conversation. I appreciate you.

[00:34:48] Victor SALSA Lopez: Thank you so much, Bonnie. I am looking forward to continuing working with DoD CDAO here and in the future.

[00:34:52] Bonnie Evangelista: All right.