A summary of our discussion during our DoKC Town Hall held in September.
Data on Kubernetes Community (DoKC) Town Halls allow our community to meet one another, share end user journey stories, discuss DoK-related projects and technologies, and learn more about community events and ways to participate.
The following provides a brief overview, video and transcript for the DoKC Town Hall that was held September 2023.
Rook – Helping the Kubernetes Storage Community Thrive
Presented by: Travis Nielsen, Senior Technical Staff Member, IBM
Rook is an open source cloud-native storage operator, providing support for Ceph to natively integrate with Kubernetes. It was accepted as a graduated project by the CNCF in October 2020 and is a DoKC Community Collaborator.
This Town Hall featured an introduction to Rook and the open source community developing it.
Watch the Replay
Read the Transcript
Download the PDF or scroll to the bottom of this post.
Ways to Participate
To catch an upcoming Town Hall, check out our Meetup page.
Let us know if you’re interested in sharing a case study or use case with the community.
#sig-operator on Slack | Meets every other Tuesday
Speaker 1 (00:00):
Welcome to the September Town Hall for data on Kubernetes. For those of you who are joining us for the first time, d o k town halls are held monthly. They’re opportunities where we invite end users to share best practices for running data workloads on Kubernetes. I want to kick off the call with an icebreaker. Sometimes we do these just to have a fun twist to the call. If you want to participate, just go ahead and chat your response here. It’s pumpkin spice holiday time, and part of our team was just talking about yay or nay. Are we in favor of pumpkin spice or are we against it? Whether it’s lattes, ice cream, candles, scents for your fabric. Just curious where folks are falling on this. I am indifferent to each his own, but I’m a personal nayyy on the pumpkin spice for coffee. Don’t mind it on a scent. So I’m just curious if anybody wants to weigh in as just a fun little icebreaker before we get started.
Speaker 1 (01:07):
I know that DIY on this call here, he’s helping with the backend he is all about. Yep. He’s chatting. He’s all about the scent. He actually said that he used it as a fabric softener scent, which is very interesting. I hadn’t heard of that, but I could see a benefit there sparingly. Yep. That’s how I feel about the coffee. I’m more of a coffee purist person, so having it as a flavor in the coffee feels kind of overpowering to me. But, all right. Also going to, while folks are still joining, I’m going to share my screen. I have a couple of slides that I want to run through before I hand it over to Travis. So just a couple of community ice announcements to tick through. We always want to take an opportunity to thank our gold community sponsors. Communities like Data on Kubernetes are not possible without the support of sponsors like Google Cloud and Percona.
Speaker 1 (02:06):
So we thank you. If you are from a company and you think you would be interested in being part of it, there’s a link here. We would love to talk to you about ways to get involved. We have a couple of events that are coming up. D O K Day is an event that’s being co-located at CubeCon and Cloud Native Con in Chicago on November 6th. It’s going to be a one day co-located event. We have about 20 talks. It’s the third year that we’re doing this. We’re really excited to be co-locating. Again at CubeCon. It promises to bring a lot of attendees. We have a lot of great talks. If you haven’t checked out the schedule, please do. We are also hosting A D O K track at the Southern California Linux Expo, also known as scale. The C F P is open until November 1st, so if you are an expert or you have information to share on data on Kubernetes, we would love for you to submit A C F P for that event.
Speaker 1 (03:13):
It’s happening March 14th to the 17th in 2024, but we’re talking about it now of course, because the C F P deadline is coming up pretty soon. We also have local meetups in this community that happen all over the world. One is taking place in New York on October 4th. There will be three sessions and two live demos happening there. The capacity is about 50 people, and we’re a little over halfway reserved for that. So if you’re in the New York area or going to be in the New York area around October 4th, we would love to have you join us, but be sure to R S V P fast, you can find that on our meetup page as well.
Speaker 1 (03:59):
And then of course, if this town hall is a great experience for you and you want to join a future one, we do host them monthly. Our October town Hall has been confirmed. We’re going to be talking with the canonical folks and it’s going to be a hands-on workshop. It’ll be a great opportunity for folks who are interested in the intersection between Kubernetes and ML Ops to join. So we encourage you to join us again next month. Let’s see here. I’m going to stop my screen share. Okay, so without further ado, today we have Travis Nielsen. He is joining us from I B M to talk about Rook. Rook is a community collaborator of the data on Kubernetes community. The D O K C community collaborator program was created to encourage collaboration amongst D O K related technology communities and the end users who are using running, excuse me, data intensive workloads on Kubernetes.
Speaker 1 (05:00):
These community collaborators like Rook are open source projects. They’re backed by a nonprofit entity like the Apache Software Foundation or in R’S K C N C F, and they cannot sell a product or service. On a more technical side, obviously Travis is going to really dive in, but Rook is an open source cloud native storage operator. It was accepted as a graduated project by the C N C F in October of 2020. It provides support for CEF to natively integrate with Kubernetes, and today Travis is going to give us an introduction to Rook. So we’re really excited to have him. Travis is a senior technical staff member of I B M. He’s also a maintainer on Rook of course, and a member of the O D F and SEF engineering team. Prior to I B M and Red Hat Travis worked in storage at Quantum and Sim Formm, which is a P two P storage startup and was an engineering lead for Windows Server at Microsoft.
Speaker 1 (05:58):
So he comes with a wealth of knowledge in this area, and we’re just very lucky to have him. We should have some time for questions after Travis has wrapped up his talk, but Travis is also up for answering questions in real time. Everybody is muted just for optimal sound. So if you have a question, please feel free to type it in the chat box here and we will monitor in real time. Like I said, Travis is happy to answer if there’s a question pertinent to a specific slide or we can save some questions for the end. We’re always available on Slack or email, so if after the talk a question pops in your mind, don’t hesitate to reach out. We often like to include those questions in the recap blog posts that we host on the DO K website. And then at the very end of the talk, we’re going to be hosting just a little bit of trivia for fun of a super short survey that folks can participate in, and the winner will get a run D O K community. So I don’t want to take any more time. You’re not here to listen to me. I will hand it over to Travis, and I just thank you all for being here.
Speaker 2 (07:08):
All right, thanks Whitney. Thanks for that introduction and happy to be here as well. Lemme see if I can share my screen and get some slides going. Alrighty. Like Whitney said, if you have any questions throughout, I’m happy to stop and talk about those, but I prefer the interactive approach just so we can mix it up a little bit. And you don’t have to listen to my boring talking for too long in a row. But anyway, I’m Travis, one of the original Rook maintainers creators of the project, and happy to talk about where Rook has come from and what Rook is, and then how we can integrate best with the community with data on Kubernetes and everything. So yeah, basically what are the principles? First of all, that fosters good community. I think we know working with Kubernetes and C N C F, there are lots of good foundational principles there. And to me, they’re just second nature because we’ve been working on them for so long. But I want to talk through what those principles are then yeah, what is r r? What stories does provide do we provide? And then again, how to integrate best with the community. What does the community definitely looking forward to your questions around or suggestions about how we can interact and keep the community thriving.
Speaker 2 (08:44):
So first of all, let’s start with some philosophy. What does it take to build a project and a community where people are interested in participating? Right? I think we’re all here because we have some basic feeling that yes, this is a good community. Yes, we want it to thrive. And so over, let’s see, what was that 2016? What are we? Seven years ago is when the R project started. That’s really when I dove into the open source space, been with work project ever since. And so what are the principles? We’ve started with, first of all, from the start, we’ve always been interested in what the community wants first, that’s maintainers. Those with merge access to the project, they’re the ones who have to have that philosophy of what does the community need? The companies we work for obviously have interest in what happens with the project, but that’s the entire time I’ve been working on the project.
Speaker 2 (09:46):
That’s not my primary concern. My primary concern is what is the community need? So I’m grateful for having had a couple of employers, actually I’m on my third officially now that has supported the Rook vision and having upstream first, second, having maintainers for multiple companies is so critical. That’s becoming a C N C F project that requires this, and we’ll talk more about governance and what that means, but having multiple companies interested in the projects and collaborating together just makes the projects that much better. Whenever there’s a project that only has a single company driving it that only has merge access, it’s just not going to thrive in the community, right? I think we’ve all seen that. We know what a difference it makes, even if there’s an open source project that’s available on GitHub, it’s just not the same if one company is behind the entire effort and doesn’t allow so many contributions or it’s so much more difficult for other companies to contribute.
Speaker 2 (10:53):
Thirdly, I’d say it requires an open mind with every issue that’s opened, every discussion item, every question. It’s just good to think about how can we accomplish this together? What will help answer questions, just have an open mind. And then lastly, governance. So the governance is really a project policy that the project is bound to, which says, Hey, here’s how we’re going to work as a community. And I’ll get more into what that looks like for the work project. I know a lot of the C NCF projects have similar governance. To become a C NCF project, you have to have guidelines that are open to the community. But even despite that, I just want to point out that the C N C F doesn’t mandate specific governance and specific project approach. It’s really what is best for that project, even with its community, because each project is self-governed and they allowed us to create our own governance and manage the project as maintainers of our own project. They don’t micromanage, the NIP doesn’t micromanage us. It’s really gives us that flexibility.
Speaker 2 (12:16):
And then more specifically, what have we done in the Rook community and some basic principles I’ve tried to hold to since starting the project? First of all, newcomers must always be accepted. All questions are valid. There’s no wrong question. At the same time, the experts or the maintainers on the project, and not just the maintainers honestly, but anyone familiar with the project, it’s so helpful if everyone can collaborate on answering questions and helping people get projects running. And then of course, all must be respectful. Again, back to this principle that all questions are open for discussion. And then specifically, we’ve definitely found some things helpful. So a slack workspace for discussions, that’s just where you get the most interactive input. There’s GitHub issues if there’s any issue or even GitHub discussions, those are just a nice place to have that back and forth. If we found GitHub issues and discussions, not as real time, that’s where we use Slack for the quicker discussions. But really we try to be responsive on all of those channels.
Speaker 2 (13:36):
And the next area that’s just foundational to a good project is commitment to high quality. How can anybody rely on your project if you don’t have that commitment? And having the clear documentation so people can get started. I love having clear documentation. I know there’s always room for improvement in no matter what documentation someone is working on, but one of R’S initial goals was, Hey, how can we simplify the experience of deploying storage for Kubernetes? And it started with the documentation that’s been foundational. Next is there’s all levels of testing, and this is just basic engineering. It’s not specific to open source. Do you have good unit tests and thorough unit tests? Do you have thorough integration end-to-end tests that for Kubernetes projects, that would mean actually run a Kubernetes instance, actually deploy the project, actually make sure the project is working, and then automating those.
Speaker 2 (14:44):
It just gives you confidence in your releases that you’re not going to have regressions. And then finally, manual testing. Of course, you’re not going to be able to automate everything, but at the end of the day, manual testing also just helps the iteration go faster and have confidence that the changes you’re putting in there are solid. And part of all that with automated testing, of course, reliable ci, there’s no perfect CI out there, I’m certain. But having reliable ci, we’ve put a lot of effort into that, the GitHub actions, and it’s just been so helpful to have all the tests running there. And GitHub finally, we had a security review when R became a graduated project. And that just gives another level of confidence that yes, people who are experts in security have reviewed the projects with maintainers, and we know that that it’s ready for production or no concerns about running in production.
Speaker 2 (15:54):
Next area, I’d say is important for a project is with this commitment to stability and testing is also a commitment to release on a regular cadence. I think a regular cadence just gives confidence for people running it that, oh, when a fix comes, we will be able to deploy this fix soon in production. If it takes too long months to have a release, then it’s just you can’t get the fixes soon enough, especially for anything blocking production. So in real, what we’ve tried to do is have our minor releases approximately every four months that gets the new features out. So you can kind of predict, okay, here’s the big new features and then the patch releases that have some minor features as well. We have a cadence, pretty good cadence of about every two weeks, and that seems to work well for our community at least.
Speaker 2 (16:55):
Okay, with that background of principles that R was founded on, get into what Rook is and how it provides storage for Kubernetes. So yeah, back in 2016 or even earlier, my team was looking at storage for really the cloud native landscape. We weren’t even really looking at Kubernetes. Well back then Kubernetes wasn’t even near as big as it is now. But we were looking at, okay, what do cloud providers provide for storage? And if you’re running outside of those cloud provider environments that have their storage technologies, well, what do you do for storage if you’re not in this? What if you have your own data center where you need storage because storage traditionally with an external thing to the cluster, but why does it have to be there that way? Why not have the storage platform deploy just like any other Kubernetes application?
Speaker 2 (18:02):
And pretty quickly, we did get into Kubernetes as and chose that as our primary deployment and target, and that’s really our only supported platform now. But then at the same time, writing a reliable storage platform, it’s not something that typically you want to start up, or at least you wouldn’t have confidence in it from the get go for production workloads because data is precious and you have to trust the data platform. So with our team, we were looking at what platforms already exist that would give us a reliable storage, and that would work well in a cloud native environment where you’ve got a distributed environment, distributed computing that needs distributed storage.
Speaker 2 (18:53):
So our goals for storage were, okay, let’s make storage natively available to your Kubernetes applications. We wanted the storage to be consumed just like any other storage that even has an external plugin. So you’ve got storage classes, you’ve got PVCs. Those are the standard ways you plug in your applications to storage, to provision volumes and mount them and things. So definitely we have a C Ss I driver that works with those and integrates with that. And then where R comes in really is for automated management, and we have a Rook operator and CRDs for allowing you to configure what the storage platform looks like or how it’s configured. So it deploys the storage platform, it configures, it provides upgrades in a smooth way, an automated way I should say. And where we ended up with was we looked at SEF as really a stable data platform. It was designed for distributed systems, our software defined storage system, but we chose sef. We made a bet on that. But Seth itself had never really been brought into Kubernetes. So that’s where Rook joined up with SEF and said, Hey, let’s bring Seth to Kubernetes platform and make it work really well in an integrated way.
Speaker 2 (20:29):
Here’s a few stats about the project, maybe to take a step back. One point 12 is our latest release and we’ve got all these downloads start just kind of your typical marketing slide. But we are happy for all the on contributions that have been made over the last number of years. And it is a C N C F graduated project. So maybe back to the storage layer now. So we did declare work is stable with the SEF storage platform in December of 2018. So right about five years now. There are so many users in production, both upstream and downstream products based on it. But yeah, I wish I knew all of the users that we had upstream. Upstream. We just don’t always, or even rarely know who all has it running in production just because of the nature of upstream. But just it’s been exciting to facilitate that group.
Speaker 2 (21:30):
Now there are so many clusters with even large clusters, hundreds of nodes, many petabytes of data that’s providing data for their Kubernetes clusters. So what type of storage, guess I should talk about that. So sef, the reason we chose sef and one of the core principles is that SEF provides really the three main types of storage all in a single platform. So you’ve got block storage, which is ReadWrite once volumes, that’s provided with sef R B D. So that’s a single pod that needs the volume for its own use. There’s no sharing in that case with a shared file system based on S F Ss, you can have multiple pods sharing the same file system. It manages that shared file system, so there’s no corruption. That’s the whole premise of S F S with those shared volumes. And then the third type of storage is with the object store and the SS three endpoint provided by sef, R G W.
Speaker 2 (22:38):
So yeah, again, the block file system and object store all in a single platform. And R configures all of that to run kind of a brief view of the architectural layers. So R does own the management of sef, so the operator deploys it as a distributed system. SEF itself is quite complicated. There are a number of demons. You’ve got the SEF Mons that are the brains ofs that store some metadata, and you’ve got OSDs that store the actual data on each disc, each underlying disc, and each node, and I won’t go into all the demons, but there are a number of demons that Rook has to deploy and automate there. The C S I driver is the next level where just like any other C S I driver, it will provision and mount the storage provides functionality like snapshot and clones, all what you’d expect from a C Ss I driver provide that. And then SEF itself provides the data layer, and Seth at the end of the day, doesn’t even know that it’s running in Kubernetes. It just knows, yep, I’m a distributed system and I’m going to provide you data. And Rick gets out of that path, or R is not in the data path, it just sets it up, mounts the storage, and then it’s all up to Seth as that stable daily.
Speaker 2 (24:07):
What environments do we run in? So R really our goal was to provide a consistent storage platform wherever Kubernetes is deployed. So if you’re running Kubernetes in some upstream Kubernetes, if you’re running in some downstream distro, Kubernetes Brook will run with it, and you can have the same storage platform that you can deploy everywhere. If you’re on bare metal, really your options are limited for your storage platform. So that was the primary scenario we were going after where, okay, you need a storage platform on top of your raw devices or local pvs. But then even in cloud environments, we found pretty quickly that, oh, even in cloud environments, there are limitations of those storage providers, whether it’s the number of PVS per node or the speed of the PBS that were limited by small sizes, A R can overcome those and also allow storage across AZs, all that, then you can work well with cloud providers too.
Speaker 2 (25:22):
And kind of to finish up the Rook intro, really our ongoing goals with the project are respond to community input. We want to know what people need with storage and then what Kubernetes features continue to come out. Kubernetes is always releasing more features, how can we build on them? And then also Seth, as the data layer underneath us is releasing new features with each of their releases. So keeping up on that really demands. There’s all sorts of scenarios. Dr scenarios, well, there’s too many to talk about, but data is a complex area and we hope to make it simple.
Speaker 2 (26:11):
A few more comments about Rook community. I guess I already said Rook is a graduated project with C N C F. There’s a process to go through to get to that state. So we started as a sandbox project back in 2018, moved to incubation later that year, and then back in October, 2020, we were happy to reach that graduation milestone. And then what do we do for governance? So for our governance, we have a steering committee, which the steering committee are the people who can vote on things pertaining to the project, and no company can hold a majority of seats. And ideally and well, and currently only one company has at most one seat on the steering committee.
Speaker 2 (27:07):
So that way voting of important decisions will take place without one company having control of it. Honestly, the votes that we need to take have been rare because the project maintainers are just on the same page and there’s no major decisions we made since graduating. We’ve been fairly stable, but just knowing that the steering committee has that cross-company vote is important for maintainers. We do have approvers, kind of two categories we call approvers, the ones that have merge access, and then reviewers are those that the approvers can merge after they’ve reviewed them. But yeah, and we update those periodically depending on sometimes maintainers move to emeritus status or sometimes we have newer maintainers who they’ve been really active on the project for a while and really they’re trusted members. So we can have a process to officially promote people to maintainers status. And there is a conflict resolution that’s defined, that’s rules for voting. Again, back to the steering committee if there’s ever a conflict, which really has been rare for voting. Again, C N C F requires a good governance for working with the ecosystem, and I really feel like it is foundational and important for any project that wants to nourish the community.
Speaker 2 (28:50):
As far as the project, again, community first is our philosophy. We have an open source license with Apache 2.0 for the R project, which is one of the more common licenses I believe with CNC F projects. And then we do currently have maintainers from four companies with ssu, I B M and Red Hat Co and Upbound, so it keeps the diversity alive for the project. I just a note about storage providers. So currently SEF is the only supported storage provider, but early in the R project we were looking at other platforms and if it would make sense to have R be a commonplace where different storage platforms came and contributed and created their operator with the work project, kind of like the operator S D K model is what we were looking at. How can we have common things that will be useful for the projects?
Speaker 2 (29:51):
But what we found is that each storage platform is so different and so specialized that there wasn’t community support to create something that was shared. And in fact, we didn’t really find the useful libraries we were hoping for. So each, what we found is operators are best with each project’s own community because those communities are so specialized. I guess. And then maybe just to come down, this is my last slide as far as what does the D O K community need or what would be useful to help and collaborate with the community? We know there’s strength in collaboration whenever we get together, whenever we work together across projects, across companies, there’s benefits to that for everyone.
Speaker 2 (30:48):
I’ll just put these questions out there. What forms of collaboration are useful to you? Is Slack sufficient? Do we have these town halls where we can get more information out? There’s focus groups. I know D O K also sponsors some monthly calls. Actually with Bart before or while he was still here, we had some calls with Rook where we had some discussions. I’d like to get those started again. And what is useful for people I think is really what it comes down to. What does the community need? How can we enable community members to collaborate more? I think I’ve talked about the platform today and what we do as maintainers, but what I would love to see is as more of, hey, here are users actually using the platforms and talking about the strengths and weaknesses of the platform. Like in Amsterdam, we had a couple of great presentations at the D O K event there. Grateful to see, it was good to see real usage of storage because at the end of the day, that’s what companies need. How do I use it? What’s an example of how to use it and what are the strengths and weaknesses? I’d love to see more of that D O K events. And I know as Whitney already mentioned, there’s the meetups and all of that.
Speaker 2 (32:18):
Yeah, the last point, I guess like I said, we love presentations by users, not just maintainers who talk about the engineering efforts. I think that brings me to the end. Any questions have come up in the chat or other questions?
Speaker 1 (32:37):
Travis, I was going to ask, this laid out a good, your presentation made me think about the folks who might be on the line who aren’t familiar with open source communities, or maybe they’re just sort of starting to work with open source communities and sort of what the structure of many of these follow. But what would be your advice to someone who maybe is interested in getting involved in Rook in some capacity as a contributor or joining the community? What would be your advice for where to get started? Is there a channel to start following? Is Slack the best place or what would you say there,
Speaker 2 (33:19):
Right, to get started with? Probably. Well, with Rook, definitely. I know for sure you go to rook.io is our website and there’s links there to join the Slack to join or take a look at our GitHub. Do I have another slide that shows there’s our website, so join the Slack for questions. Look at GitHub for issues in GitHub. We do label our issues as good. First issue a label for that, just so newcomers can look for simpler issues to fix and that’s a good place. Or just joining Slack and saying, Hey, what I’m new, how can I get started? I think whenever getting started, the first thing I’d recommend is go walk through the documentation and try running the project. And as you run the project, you’ll start to understand it. And then when you start to look at issues, just having some idea of how the product works, the project works will help get started too. So it’s got to be a very hands-on thing to understand what you’re looking at.
Speaker 1 (34:33):
Speaker 2 (34:38):
Yeah. I guess as far as Slack, maybe I’ll point out, we do have a couple of main channels, just our general channel, the dev channel, C S I channel. There’s the dev channel. It’s a good one for people looking to contribute.
Speaker 1 (34:56):
Okay, great. So rook.io, there’s a lot of resources on there, many different Slack channels to participate in, but great to know that there’s also a categorization, I guess for lack of a better term on GitHub for folks who might want to get involved technically speaking, that they can kind of identify first issues. It sounds like that might be good ones for newcomers to get their hands on.
Speaker 2 (35:27):
In our documentation, we have a developer guide that is how to get started with your work, how to open a pr, what our guidelines are around opening those. So look for the developer guide in the book documentation.
Speaker 1 (35:43):
And to your point about one of the benefits that we hope to provide in this community is really sharing the use cases and the end user stories, the ways that people are using different technologies in the communities. And so if there are folks on the call who have a use case or they have something that they’re working on troubleshooting, we’d love to hear from you. That’s a great point of collaboration to sort of know what folks are working on and maybe you need to be put in touch with someone, or there’s another community that you’re sort of trying to work with and you’re not exactly sure where the entry point is. These conversations are great starting points for that. And the D o K community really is hoping to be that resource of connection points so that we can get more stories out there and more use cases. And the events, like you mentioned, Travis, are one of the ways that we can do that, really showcasing the how and the technologies that people are utilizing. So we really appreciate you taking the time to come on and telling us a bit about Rook and giving us an overview there.
Speaker 2 (36:57):
Speaker 1 (37:00):
So of course, like I said, if questions come to folks following the call, watching the recording, just thinking about this further, please reach out to us on the D O K Slack. You can email us, you can find us in a myriad of ways. You can go to the website. Anyone on the team can point you in the right direction, so please don’t hesitate to reach out. We do have a really quick quiz that we were going to share here just for fun to kind of test people’s trivia. Like I said, the person with the most points will be eligible for a run D O K community T-shirt. We run these quizzes via mentee, which I think diodes, I’m hoping you have that side. I’m just realizing. Okay, there you go. He’s on it. So there’s two ways you can join. You can either scan here with the QR code, which I find to be the easiest, but you can also go to mentee.com and enter the code 4 7 8 9 6 4 7 2. So I’m just going to give everybody a minute to join on and then we’ll get this quiz started.
Speaker 1 (38:16):
Just see how much information we know. All right, I got a couple players in there. All right, here we go. In the 2022 D O K report, what was the reported top data workload on Kubernetes? Got a couple options here. See what folks think. Databases. Persistent storage was one that folks thought it got tricked. It was databases. Indeed, I like to show the leaderboard along the way. Just keep things nice and competitive in a friendly way, of course. All right, question two of three. Short quiz, as I promised on the top data workloads on Kubernetes, which made the biggest jump in popularity, databases, analytics, AI and ml, persistent storage, streaming and messaging C I C D.
Speaker 1 (39:56):
Nice. It is indeed AI and ml. We’re seeing that rise year over year. Okay. Let’s see. Did it change our leaderboard? That’s a close one. All right. Last question here. Where and when is the next D O K day taking place this year? New York virtually in October, or co-located at CubeCon in Chicago on November 6th. Test your listening skills. Yes, although we are having a New York meetup on October 4th, and if you’re in the area, we encourage you to join again. There’ll be a couple of sessions and live demos there, but that will not be the official d o k day. D o k day will be taking place at Q con and Cloud Native Con in November. Okay. So we had a very close competition here, but Jacob, you are the winner. So we will be in touch following this to get your information and make sure that we can get you a run D O K T-shirt. So thank you everyone. Yes, I love the enthusiasm. Thanks everyone for participating. And more than that, joining this town hall, again, we have these town halls on a monthly basis. You can follow along the calendar of events on our meetup page. The recording of this town hall will be posted to the D O K YouTube channel. We have a dedicated playlist,
Speaker 3 (41:39):
Speaker 1 (41:40):
Labeled for town halls. We’ll also be posting a recap blog. And like I said, if you do have any questions related to Rook, please reach out to us and we can connect you with Travis or the Rook team. And again, we thank you for your time. Enjoy your day. Bye all.