Maintaining the Slack voice and tone at scale

Bryan Montpetit 00:00 That being said, we have our next our next session coming up momentarily. And it is maintaining the slack voice and tone at scale, which is presented by Annika Griff. Again, I hope I didn't massacre her name, so she's gonna be able to correct me momentarily when she pops in. So she's just connecting now I believe. And I think we have her with us. Okay. Hello, how you doing? Anca Greve 00:26 I'm great. You're doing great with my name. Bryan Montpetit 00:30 Thank you, I was, I was terrified that I was gonna ask you your name. So I'm very happy that I didn't do that. And Aika is the senior localization manager at Slack. So and this is great, because I feel like Slack came out of nowhere, you know, took the world by storm and, you know, provided us with this new way of, of collaborating with our colleagues. And you know, I think based on that, that fact alone, everyone's going to be all yours. So, with that, what I can do is I can hand it off to you. I'll pop back in with about five minutes left, so we can go over some q&a. Anca Greve 01:03 All right, sounds good. Okay, plain here. Hopefully everyone's these this Bryan Montpetit 01:09 do indeed. Anca Greve 01:10 Okay, still good. still see it? Yep, absolutely. Excellent. Okay. Thank you so much, Brian. Hello, everyone. My name is Anca Greve, I lead localization at Slack. I'm the senior manager there. And today, I will be speaking to you about the slash voice and tone scale. So let's start off. Talking a little bit about the company. I'll speak about how we maintain the voice and tone across supported languages. To maintain brand trust with international users, including describing our processes and systems that actually enable us to carry out the work work against our SLAs and our lunch don't know what is slack. I want to just take a couple minutes to introduce you to Slack and tell you a little bit about our business, because I think it'll be important context to some of the things that I will touch on. Lack helps people work together collaborate easily online just as easily as they do in person. It's designed to support the way we naturally work together making collaborating with people online, easy and efficient. As we often do face to face. We have, the way we describe our product is that it's composed of four parts. We have the core product, which is the screenshot that you see on this slide. This is where teams connect where people and the apps that they work with every day get work then we also have the slack platform. So slack is an open platform that allows you to configure and customize your own unique work needs without leaving slack. We also have Enterprise grid enterprise grid is designed to make everything people love about Slack more powerful for larger organizations. It connects unlimited workspaces for each department office location team, and offers additional messaging capabilities that are shared across the entire company. And we also have shared channels and shared channels bring the power of Slack channels and apps to the work that happens between companies. So share channel provide a secure common space that makes working with external parties like maybe a translation vendor, easy and seamless. And we at Slack believe that share channels will actually replace email as the primary way that people communicate and collaborate at work. And just a couple of stats here we have over 750,000 organizations on either free or paid plan with over 122,000 paid customers, including 65 of the Fortune 100 companies. Half of our diu or daily active users are outside of North America in more than 150 countries. And we also have 2165 employees across 17 offices in India, in AIPAC and in North America. All right, so let's let's dive in the to the actual meat of this presentation, which is the localization part. So threw up a little bit of a timeline to just give you all the context of how I started localization at Slack. I started the program in 2016. As I just mentioned, we actually always had a very large international presence, and it was just increasing back then it obviously persists today. And so we wanted to start localizing the slack Help Center into four initial languages, we wanted to start with French and then do German, Japanese and Spanish, the company was still really small. And maybe some of you have had this experience where you kind of wear multiple hats. And so I actually started my job by translating the health center into French. While at the same time I started hiring people for my team, I on boarded our translation management system, and continued to make the case for what we asked what I thought we needed to to do localization well at SLAC. This timeline takes you a little bit further out to 2019. We actually launched our first wave of languages in 2017. And then I kept hiring, you know, the people that allowed me to to scale and support the company's growth, which I will touch on a little bit later. But first, here is some view of our help center the front page of it. The decision actually, as I mentioned before, was to hire professional linguists full time, though, the reason I wanted to do this was because I knew the importance of offering a white glove treatment and markets that could really boom for the company. And I also realized early on that Slack has and you know, have them still has a competitive advantage and how we address our users. So ensuring top quality was very important to us. As you saw on the previous slide. Eventually, we did hire the four linguists, we needed to support not just help center launches, but the product launches. And to dig further give context as to why I really thought that in house full time role was needed from the linguistic perspective. One of our principles in our Slack style guide is to be clear, concise, and human. And it was this human element with pervasive culture in Slack to pay really close attention to how we address our users to how we support them, that signal to me, Okay, with this initial localization, I really want to gain the users trust and trust, you know, from people who have never heard of the product. And I wanted to meet those people where they were. And I wanted SLAC, to be familiar to them and speak to them in a way that that they understood. So this sort of philosophy and structure served us really well for a while. But today, the localization team actually looks very different than what it looked like in 2016, and 2017, I have reorganized the team into three pillars, projects, systems and quality. And I did this because I realized that and so the company just grew incredibly fast. Like Brian said, we kind of came out of nowhere. And I needed to figure out how to better scale the team to continue to help the company grow, and to continue to remain the hub, where all translation, all translation requests came through. But that we could maintain that consistency and that brand promise. As linguist, the team wasn't able to scale as fast as the company grew. So I was also really afraid of burnout for my team. So again, the reason we transitioned into these roles was to really intake process and deliver the work at a quality that we were really proud of. And there's this mindset at SLAC, of just constant experimentation that we have, especially in my department and customer experience. And this leads us to be able to make these types of decisions when one thing no longer works, we change it, and we keep keep iterating on that change every couple of months until it works. And then we probably change it again. Because the company is just growing so fast at this constant iteration, this constant experimentation is key. And I always say to my team, many of whom, you know, many of these people, you see, we're not in in these roles before. So I always tell them, It's not about what you know, it's about whether you can figure it out. And I think you know, that's true for our world right now. I digressed a little bit, but if you kind of think about where we were in the beginning of the year versus now, I'm sure you all have had to adjust to match the needs of your company and your customers in the market and, you know, the vertical conference, so it's a very different place. So we adjusted as well to match the needs of our company. But one of the dilemmas we had as we reorganize the team How do we maintain top quality without in house linguists. Quality is really important to us because in addition to meeting our users where they are, it's how they recognize us. It's how they recognize our brand. And so we wanted that brand consistency, so we don't lose our users trust. And today, we do remain the hub for all translation requests slack across all supported languages, partly because we were able to shift our staffing model here. So how do we do it? So first of all, being the hub in addition to maintaining that consistency, and brand trust puts us at an incredible advantage just in terms of in terms of the work that we can see inside the company and the processes that we can follow. We currently support seven languages, and power every single team that needs stress. So every single team has slack that needs translations, their requests go through my team, which translates into 10s of 1000s of words that we have to process per week. So let me show you how we actually do it from intake to quality review, and then delivery. So here is a sample request. We intake all translation requests in a channel in Slack, it's called leave translation, you can see it there at the bottom PLC dash translation. Even if we've been roadmapping with the team or gotten a notification of an upcoming project, we still request that every single person who has a translation request, go through this process. And the reason for that is it gives us volume, it gives us ability to predict spend and other data that we otherwise would not get. So anyone that slack, can use a workflow that we've built in this channel, essentially fill out a template and request a translation. If you haven't used workflow builder, you can use it in Slack, you can read our Help Center article, it is amazing, it basically allows us allows you or us to build, you know, this template that people can just fill out and give us the information that we need. And we have two workflows in this channel. One is for engineers, one is for non engineers. And this sample is a sample engineering request. We also take you a little bit through we basically have a priority system for our SLA is with red, blue and white emoji circles. red circles are usually urgent. So 24 or 48 hours, blue are three to five business days. And then White is non urgent. So this request was a blue so that means that they probably wanted something like this back in about three to five business days. So one of the project managers on my team, who is on triage goes into this please channel, someone is there to triage on a daily basis. And they literally put eyes they put the emoji eyes on new requests. So that signals to the requester. Okay, somebody is in taking my request and it's being processed. Once we put eyes on the request with the emoji, we create a JIRA ticket. And we assign it to the correct projects pillar owner. So we know what category each person on the projects pillar owns. And we assign it in JIRA, we assigned the due date and make sure that if we can't accommodate it for any reason, we talk back and forth to the requester. We do this all in Slack in this channel. And then we we send it off. So each project manager on my team has a JIRA board with all of their projects. So it's really easy for them to see on any given day what the priority is and what they have to focus on. And then when I say we create a JIRA ticket from this request in the channel, you would actually have a little button that you could click, and it would allow you all within slack to just click on the button and automatically create a JIRA ticket. And this is because we have our JIRA integration in the channel. So we just click a button, it pops up a forum for us to fill out with all the details and then assign it to the right project manager. So slack integrates with JIRA, and many many other tools like Google workato Zendesk, so it's really easy to stay in Slack and do all our work in one place rather than go back and forth between apps to traffic. So we intake the work we process it, we deliver it how do we actually measure quality? These are actual quality statistics for us for German, French and Japanese quality in the past fine with with the individual linguist, it was very, very manual and unscalable we were doing the work in house across all languages, and just the volume made it impossible to keep up. And so we didn't even have linguistic metrics, we didn't have any alignment on linguistic initiatives. And now we no longer manage languages, we manage quality. The difference is that language management is word based, whereas quality management is metric space. So these are, again, actual metrics from March to August. And the way we actually run our quality program is we don't rely on specific people, but a system of checks and balances, where each of our branches that we employ to do this quality evaluation checks the quality of the other branch. So we work with three types of vendors to accomplish this one for the regular translation editing workflow, one for the linguistic quality assessment, and then writers and consultants on an ad hoc basis to do additional checks, copywriting and transcreation. The way we perform linguistic quality assessments depends on the project as well as the timeline. We do everything we intake bugs, we do customer and internal feedback we do in context review and post delivery quality checks. And the most recent part of our program, and the continuation of that linguist base detail oriented voice and tone focused philosophy that we started with, is the post delivery quality evaluation. It is these metrics that you see here that ultimately give us a gauge of how are our languages performing, how are our translations performing? So the way this works at a more granular level is we select content across our various languages. So I think I hear my echo. Okay, maybe you're fine now. So the way this works is we select content across various languages and projects at a random cadence. But right now, we're trying to make that a bit more like regular. So we actually do it across these three languages one time a week, and we go between our product on help center the website, and we just kind of take a little bit of a sample of each. We leverage content qual to do the evaluations and beyond for the their linguist tool. And to measure quality we we've developed an era typology, and it's based on industry standard seven category error matrix. And then for severity levels, billing was slog into content flow where they perform the evaluations on the selected piece of content. And that's how that part works. We actually recently introduced more grading in our air typology and develop then divided it into five segments, from excellent to poor, rather than pass or fail. And that gives us a little bit more nuance within these scores to really know where we're working. So, so far, we've had a lot of really good quality content. But we are developing a process to look out and address If anything falls below the quality standard that we've set. And this process will also hopefully ensure an arbitration step actually between our linguists who are evaluating the translations and the LSP, who is doing the translation. Alright, so here is how we do it. These are some of the tools that we use in our toolkit. Our LSP is Meraviglia. Our translation management system is martling quality evaluation software's content quality, and our linguist full is source through beyond. We also have as I mentioned, numerous integrations Jira, workato, Trello, that help us run things from within Slack. So JIRA helps us with project management tracking. Trello is where we actually do our CIA's with our LSP. And then workato provides us with an automated notification system for any kind of comments that the translators might make. In Maratea, software and Symphony we actually get those notifications in Slack, which saves a lot of time. So every system we chose, we chose for a reason, and it was to scale efficiently and help the company grow. For instance, with content quality really needed a piece of software that we could just plug linguists in and then they It could be free to do the evaluations at their own pace. So what's next? Well, we had a lot of activity in March, we grew our connected users base in a matter of days to 12 point 5 million simultaneously connected to users. So we saw a lot of people begin to continue to use Slack during this pandemic. during weekdays, the cumulative cumulative number of minutes of active users of slack by all users globally now exceeds 1 billion. So I think our challenges are going to be as we grow our presence globally, as we grow our our in market presence with offices and with people on the ground, developing systems that can continue to ensure we maintain that brand trust, that linguistic consistency and allow these experts whether they're sales or marketing or something else, do their job, but also stay in touch with with, because we do process, all translation. So we want to make sure we can keep doing that to maintain consistency. So I have a couple of takeaways here, too, that you can kind of remember. We leverage our app share channels and integrations to automate, work and communicate with vendors, whether it's our software providers or our LSP, or other vendors. We also try to automate as many things as possible, including how we intake projects, and how we measure quality, to be able to scale alongside slack. And that's all I know, I think It's question time, I'll stop share Bryan Montpetit 21:58 this video. Thank you. Thank you so much for that, that was great. And as a result of the being great, we do have questions. So that's we'll just hop right into it. So we have first one, it says you've mentioned swift turnaround times two to three days. But do you have specific criteria for splitting files? Or do you simply split files between different translators on a routine basis? Also, what quality and process controls do you have in place to ensure that the work different translators? Sorry, that actually jumped around on me? Different translators is cohesive in the final version. Anca Greve 22:33 Okay, so do we split files between different translators? And how do we make sure quality and process controls, basically get all these different things into one place and have good quality. So we sometimes do have to, if we have a very extensive project, we try to give the LSP a heads up so that we don't have to run into a situation where multiple people are working on the same thing. If we do have that, though, because we do have large projects, we try to segment it in a logical fashion. So like, let's say we have a doc, somebody will do the intro, somebody will do the body, somebody will do the ending. And when we do have something like that, we always try to get a brief from the content owner saying What's this project about? How do how do we want to translate it? What audience are we targeting? So we try to give as much context to the linguists as we can, so that they have that big picture of okay, I'm seeing a random sentence, and they kind of know what it's connected to the Bryan Montpetit 23:46 grid, and then she's gonna say, I think that was the first part. Anca Greve 23:51 Yeah. And then I think the second part was, how we how we keep cohesive quality? Yes, precisely. Okay. So it's hard. It's really hard because we have to move very quickly. But that's why we have this especially on copy that's highly visible, or, or a lot of times, this is marketing copy or brand copy, we actually do a lot more post delivery, quality evaluation, it's more extensive. We schedule it like it actually it happens at a more granular level. For these types of high visibility projects, thankfully, we we often have a bit more notice than for like, I don't know, 10 engineering strings. So we do in the schedule, we do ask the stakeholder internally to budget time for for the quality assessment even before delivery. So we have two ways of doing it either before delivery. See if they have the time or right after delivery. And that would be that would be the way that we try to maintain cohesive cohesiveness and quality. Bryan Montpetit 25:11 Perfect. Thank you. Another question for you. Do you have quality evaluations on an ongoing basis? So with that, do you evaluate all types of requests content types? And what's your baseline? So what's the lowest score that would constitute a pass for example? Okay, Anca Greve 25:28 so the post delivery quality evaluation program is relatively new. So we organize reorganize the team last year. So even the reorg is relatively new. And so we started this program in March. So we're still developing it, which is why we only have three languages rather than our seven languages that we support. And which is also why we pick up little bit from the Help Center and the product and the slack.com site. Because the Help Center addresses the different types of audience, the product, sometimes it it's more broad. And then the the.com site is probably four users who are visiting the site for the first time. So we try to select from three areas in three languages. And again, this is because we just started but I think that also gives us a good view of different customer segments. Bryan Montpetit 26:32 Okay, and with respect to the lowest score, is there something specific you'd like to communicate? Unknown Speaker 26:39 So our lowest score right now is 99%. So if something is under 99, or, you know, not near perfect, we we really tried to dig in and see why and understand why. So, again, like we were still adjusting a lot of these, we just implemented the the five point, you know, excellent support rather than pass fail. But I think this is a testament to how great our translation teams are, we do a lot of training with them as we onboard them as to what slack is and how Slack sounds. And they're able to carry this through to a level that we're really proud of. Bryan Montpetit 27:22 Drastic, well, thanks for that. Okay, and again, the question is just coming in. So I'm sorry, if I'm gonna pardon you, I'm just, I just know that they want to be able to get through the ball. So. Right. So I don't know if you'll have the answers for this one. But irrespective of how big is your developer community? And how easy was it for you to onboard them within the slack JIRA integration? Did you face any particular challenges? Anca Greve 27:48 Robert, that was not my team. I can't speak to that. But I have your info. And I can connect you to the person that can will say our developer community is very large and global. But I can I can connect with Robert offline and give him some stuff. Bryan Montpetit 28:10 Right? For clarity, you mentioned that you normally deal with seven languages. May I know what those languages are? And do you have a plan to include Indian languages? Anca Greve 28:20 So the seven languages we support are French, German, Japanese, Spanish for Spain, Spanish for Latin America, English, UK, and Brazilian Portuguese. We, I would love to continue expanding. I'm not the person who makes those calls. There's a lot of internal collaboration to decide what markets we enter next and what languages we localize in. So unfortunately, I want to say yes, we of course, want to support all the languages because, you know, I grew up speaking a language other than English. And it's so nice when you can have a product in your language. But unfortunately, it's not my call to make Bryan Montpetit 29:06 very diplomatic well answered. So very good. And this one, I think everyone's ears are going to pick up for the next question, which is with respect to is there any opportunity or opportunity to accept cooperating with external LSP besides our Ws? If yes, what's the process in order to initiate the cooperation? Anca Greve 29:29 Right? So we have just started collaborating with Mr. Avia after a pretty extensive RFP request for proposal process. So currently, unfortunately, no, we're still getting to know each other and, you know, developing our relationship and we're seeing really good work from them. But, you know, Slack will continue to grow and we don't know what the future holds. So maybe let's talk in a couple years. Bryan Montpetit 30:00 Very good. All right, that's good. I had another question that came in. And I was wondering the the structure of your team, the one that you put in place? Did you model this after another company structure? Or was it something that you created? And if so, why? Or if not, no. Anca Greve 30:17 I think our industry it's quite small. So I'm sure there's probably someone out there that's like, that's what I kind of did. I have talked to so many people from from peer companies who have advised me on the various parts of what I would need, like, you're going to need project managers when you start growing like crazy. And they were right. So I don't know for sure. I just know that, you know, the advice I got from from our colleagues in the industry was was great advice. And it's working for us. Bryan Montpetit 30:50 Fantastic. All right. And I mean, with respect to your structure and whatnot, and obviously, things are moving well, according to what you've presented today. How do you actually determine the ROI for localization investments that we've made? Anca Greve 31:05 Yeah, it's, you know, ROI, like this is the question. We have better metrics now, working with Maravi Actually, both volume metrics, and we're able to split projects by the department. And so we're able to, to work with the department and tell them, you know, here's what process for you guys, what we prompt the three guys. And, you know, I think we're just now engaging in a conversation where, a couple months after we started working with with Monrovia, we have those metrics to present to them. But we haven't yet gotten to a point where we're able to connect the dots between the work we do and the work they do and kind of see truly what you know, take out all the other things that for instance, marketing does and just look at localisations impact in in a certain area. I don't think we're there yet. But we're definitely working towards it. Because I also think that's really one of the ways that you can say, I bring value. This is why you do and work through me and I consider our team, a service provider to the rest of the company. And I think that ROI is key. It's just a tough thing. And we're still working towards towards that that goal of getting better metrics ROI. Bryan Montpetit 32:24 Absolutely. It seems somewhat elusive and trying to calculate it. So it's I do I do understand it's not easy at all. Anca Greve 32:31 Definitely another question I asked everyone I talk to in localization, how do you do it? Bryan Montpetit 32:38 Yeah, it's definitely I think, one that. I mean, everybody has an opinion on it. Right? So I think it's, you become you become well educated in terms of just by asking everyone, and then you got to do what you can put into place. I think, you know, within reasonable time, and then with what you have so definitely. One other question about, you mentioned Spanish for Latin America. How did you decide which Spanish of Latin American Anca Greve 33:09 and we choose? Our initial launch in 2017 was Spanish for Spain, and that I think was based on usage metrics. I think that as we grew, we