Increasing content accessibility for Multilingual Communities: challenges, processes, needs

Clark Hayes 00:06 I want to welcome everybody who is watching us today to our panel, introduce our panel which is increasing content accessibility for multilingual communities, challenges, processes and needs. I am excited to introduce our panelists today we have three language access professionals in the public sector. What we will be discussing today is some of the similarities and differences between public sector and private sector and some of the unique challenges that governments face when it comes to translating for their multilingual communities. So I will let Suzanne Peggy and Santiago introduce themselves. Suzanne, do you want to go first? Susanne Brunhart Wiggins 00:51 Sure. Thanks. So I am Suzanne Wiggins, and I am a senior Information Technology Specialist for Montgomery County, Maryland. And in that role I have I work on websites for various agencies and work units within the county. And relevant to our discussion today. I've been managing our COVID website for almost the duration of this time. And I have done quite a lot of work there with multilingual content. Clark Hayes 01:23 Perfect and Santiago. Santiago Torres 01:27 Thank you for having invited me in having us here for this panel. My name is Santiago Torres. And I use he him pronouns and I manage the Language Services Unit here at the New York City Mayor's Office of Immigrant Affairs. So we, we rent their services to Moya, and about 20 Different Metro offices to City Hall, and just the breadth of content, digital and print. So our target audience is that 2 million in limited English speaking, English proficiency New Yorkers. So that's about the fifth if it was a city about the fifth largest city in United States. And so there are roughly about 600 languages spoken in New York, that's a couple of smaller languages that fall into other families of languages. So we try to cover as much as possible. So during the time that I've been here, we've expanded from about 18 to 65 languages. So and that's across materials, and our website is localizing the 10 local or 30 languages, which we'll get to. Clark Hayes 02:34 Perfect and Santiago for those of us who are not native New Yorkers Moya is, is what organization. Santiago Torres 02:41 So Maria is Mayor's Office of Immigrant Affairs. Clark Hayes 02:46 Perfect. And Peggy, if you want to introduce yourself, Peggy Liao 02:50 yeah. Hello, everybody. Thank you for having me. My name is Peggy Liao. I am with the City of Seattle office of Immigrant and Refugee Affairs, and I'm the language access program and policy specialist. We are not as as big as New York City. But in the school system, we have about 150 languages that are spoken in Seattle, and about 25% of our city residents there speak languages other than English at home. And we really because of COVID a lot of departments and really city governments and then those who County, we work with public health a lot. Realize that translation is a challenging area. And then we really need to be able to have more control over how fast and and the translation quality that we can get. So we recently introduced a translation management system or we're using SmartCAT to help assign translation project from city departments and be able to work with local translation vendors and intern individual translators more efficiently. And we I'm sure we will talk a lot more about that later. Clark Hayes 04:31 Perfect. Thanks, Peggy. and welcome again, everybody. I'm Clark Hayes account executive at smart cat before we get started, so government one thing I want to preface government organizations have similarities and differences from private sector, but they're unique in that they use taxpayer dollars to serve their constituents. So everybody runs into the same three challenges turnaround time quality, cost. Governments are unique in They have to be very resourceful about how they use their money and how they approach those turnaround time and quality issues. So those are the three main topics that we'll discuss. And hopefully Suzanne Santiago, and Peggy will give us some great insight into what governments do to manage that. The first one is turnaround time. And since COVID, governments have had to respond very quickly to public health information and have had to get content out to the communities and update that content in real time. So one of the first topics, I'd like to discuss how we handle that fast release, changing content across all the languages that we work in. So the question is, is simple it's just what, when it comes to turnaround time, what obstacles challenges do you face in your individual roles? And how do you approach them? What kind of solutions have you come up to deal with them? Does anybody want to go first Peggy Santiago or Susan. Susanne Brunhart Wiggins 06:06 I'm happy to take that one. It has been my bread and butter for some time. Very familiar problem. Unfortunately, we challenge you know, let's turn it a little bit positive. So when updating COVID content, sometimes that has changed multiple times in a day even. And so we present some of our content in seven languages in English, and then six other languages. And so my challenge has been submitting that for translation and having the English content, which has been like that's my, my primary of what I compare the translations to having that change before I even get the translation back. So that then becomes difficult, because not only are we juggling the versions, but they're out of sync before they even get get back to me. And that's not because the translation team is working too slowly, it's more a reflection of how quickly the material information is, is getting updated. And so one of my workarounds has been or how I deal with that, and I'm sure that we will come back to this topic multiple times during our time together today is that when I write the English content, or edit the English content, I really do it with an eye on plain language in the hopes that if that language is as clear and as clean as possible, that it will be easier for somebody who is less proficient in English, to understand in English, and if they ended up using a machine translation tool, that when they put that language in there, the English language that it will come back as the more successful translation. Clark Hayes 07:48 Perfect. And I think Peggy, you had some comments along those lines as well, maybe if you want to? Yeah. Peggy Liao 07:58 I definitely agree with Suzanne, I think plain language is a really important component. And really, everyone should write in simple and plain language. And another area is we're able to work. So when the pandemic when everything happened 2020 March, I personally was just like borrow to King County Public Health to just work with them directly. So we have that direct, firsthand. communication materials, confirmed by the leadership, we can quickly review what needs to be publish the day of an early morning. And my role is really to just take key component really just need to let people know what to do. Now, like a we tend to write and very long blog posts, say what happened yesterday, what happened last week. So today, we have to do X, Y and Z. We're able to just take what needs what you need to do today, and then translate that component really quickly. And so that's one area and another area that I mentioned earlier that we are really thinking about what other ways that can help us quickly turn around our translation project. So instead of working with translation vendors, they're bigger and then need more time one or two weeks. We basically just contact people who are corps certified that people would know or professional translators and then invite them to be to be a group of COVID Responding translators, and then we just work with them directly. And back at that time, we still communicate via email and everything was done that way. But we were also able to translate into 710 languages within 24 hours. So that was the two approaches that we took here and Seattle and King County. Clark Hayes 10:30 Okay, nice. And Suzanne, sorry. Just to get a sense of scale, How many languages do you work in for those projects? And how many individual translators Are you coordinating with? Susanne Brunhart Wiggins 10:41 So we have seven languages. So English, and six others. And during most of the COVID time, the county pulled together a team of volunteers. So in turn people who had internal capacity to work in multiple languages. And so there was a group of people working on each language, some languages had more depth than others. Spanish it was easier to find people for than something like the Amharic team was the smallest team. So it really depended on people's availability on that day, of of who was able to do what. And so that was another challenge, or is another challenge is that translations come back at different speeds. And so my instruction to the team is always don't hold the translation, give it to me, the moment that you have it, and I will be delighted to receive it and to apply it into the the page that it needs to go on. So that we're able, because the reason that we're doing this is to provide equitable access to the information. And so I don't want to hold it, just because we don't have all seven languages. What I do is I do if I feel like that information needs to go in other languages, at the same time, I'll put the plain language. So we'll have a mixed language page, which is not ideal. And then as soon as the translation comes in, then I'll replace the the English part so that the page as quickly as possible can return back into the language that it's supposed to be in. Clark Hayes 12:23 Perfect and Santiago on your end, when it comes to turnaround time, what what kind of challenges do you come across? And how do you approach them? Santiago Torres 12:32 Yeah. So just to take a little step back, on how to paint a picture, how things were when they came in, really, there was everything was an ad hoc, there was just like, a shared drive. And they said something Oh, here you go. And like some contracts that were drafted 10 years ago, or 10 years ago in 2010. With some language that I mean, regarding turnaround times, I mean, the language there, so you know, then the contractor most delivered things in CD ROM, by this date, you know, it was like that. So very much an acronym stick. So one of the first things we did, it's like we centralized services back then, you know, audited vendor performances, try to move away from, you know, doing this work with just with bilingual staff, because a lot of them have a lot on their plate already. And so, and we just went on this long education and training process, by the time the pandemic hit, we had already been successful at securing some good contracts, particularly with one vendor for translation, a New York based vendor. And I remember clearly the day before we went to lock down having a meeting with them. And you know, they understand the work. They work with a lot closer institutions here in New York, and across United States, they work with other city agencies as well, there's some all LSP and so they once we were all affected by that, so they were you know, as well, that we weren't having teams across the world and so forth, even this global pandemic. But I'm in one of the meetings, one of the representatives said, you know, we'll do anything because as is our duty as New Yorkers. So what we did is, again, just for the meaning of pandemic, they assembled a team of it was 25 languages that were the first response, the rapid response languages within the beginning of the pandemic, so the first four months of the pandemic, and those we, you know, social media, notices, infographics and so forth, that we were able to deliver between three to 24 hours depending for language. So they had a translation that translator and editor, bird language for the 25 languages, about three or four account reps in the US as well moving information on so there was really about nearly 60 people on call during this seven days a week during the first four months of the pandemic. So that was basically it was an issue of vendor management and ensuring that everybody was on board. The other is that by that point, we have already built out a Spanish unit. So we already had some teams. In Spanish we are going to use trials and multi term. So we have some glass recent times and but then again, terminology was evolving as well. So we had to adapt and do some research and so forth. So a Spanish, we were able to deliver immediately any, any types of changes in versions or edits or updates, we were able to do those instantly. But also, one thing that happened with timing is that we had been piloting working on the website localization project with Marlin for several years. And it was really slow, that by the time the pandemic hit, and again, it's complete timing, we were done. So when we were able to present this to our commissioner, really executive teams and so forth, because they're like, how are we going to do this digital content? How are we going to move this dynamic content faster? Well, we actually have this thing we just waiting to show it to you. And then we immediately created a COVID resource site that was updating constantly. And at the beginning, during those first four months, we have a 25% of web traffic was coming from the multilingual sites. And so it's, it became a very useful tool, we were putting, you know, the link into infographics, it will share with our communities, you know, WhatsApp, and WeChat, and so forth. So, again, it was just coincided a lot of this rapid response and improving turnaround times with work that had been done for the last two years. So that we were perhaps one of the few teams that was ready for something completely unexpected. But still ongoing process. Naturally, there's always a challenge. Clark Hayes 16:26 Didn't I think that's similar? So the website content is is interesting, and Suzanne has similar experiences there, I guess on you're in Santiago, is that also going in 24 languages? And just to get a sense of scale, as well? What's the how many times a day or week? Are you updating that content? And what's the average kind of size of a project going in those languages? Santiago Torres 16:47 Yeah, so at the beginning, it really has changed over time. So the we translate into 10 languages, the 10, local authoring languages, I mean, include everything from Arabic, Bengali, Korean, Spanish, we do, so that you see the breadth of the type of scripts and the challenges of Right to left languages and so forth. And this madaline SmartLink platform, you know, help take care a lot of things that our team struggle with, you know, some other localization back end internalization, work that needed to be done. So that was handled. At the beginning, we were doing. We're doing large update, I mean, if not completely read, launching new sites, you know, that we're anywhere between 10 to 20,000 words, and we can any of those will be done in a say, maybe a week, week and a half or so, for dynamic content, you know, we can turn around in less than 24 hours. So some of them if it's just as a short, like just sort of messaging just in the few hours now that the website is not, because we also go into administration change. So the programs and other initiatives are changing. So right now, those updates are only like once a week, and a lot of it is just again circumstantial. But now that we have, you know, we have a new commissioner, a new executive team members, new mayor, so forth, as programs and messaging and something that's there, because there's always something happens here in New York, it would likely have been that usage will increase once again. But the infrastructure is there already. So pretty much ready for whatever comes? Clark Hayes 18:25 Nice. And Suzanne, on your end? How often were you updating that content on the website? And what did that process kind of look like for you. So Susanne Brunhart Wiggins 18:34 I was updating a lot. And I'm still updating thankfully, the pace has lessened a little bit. But really, just yesterday, I made a change to one of our translated pages. And so now I need to see how I'm going to handle that moving forward, because our process has changed a little bit. So the English content, sometimes I was changing multiple times a day. In the height of their pandemic, what I found myself doing was at the end of the day, like, you know, late at night, I would think about what has changed today and what is the stable version, and then prepare a file to send to the translation team that they would then work on, not when I sent it to them, but it would be ready for them the next morning to for them to work on and get back to me as soon as they were able to do that. One difference that I wanted to elevate that I feel is different between the web content and maybe the more traditional way of thinking about translations is that in my world, sometimes it's just a sentence here or there that changes and so I'm not like the whole text is rarely ever ready at once. You know, initially the whole text is ready and then it gets prepared and sent over and I get the whole thing back and we put it after that it's more like oh this element change. So sentence here sent and spare. That is actually for me more difficult, because then when I get the translation back, when I get the product back, I need to figure out where it goes on the page. So we now have a translation management tool that allows me to send HTML, and they work on it, and it comes back as HTML. So that is, I was so overjoyed when that capability came into play. Because otherwise, I would have to cut and paste. And that actually is physically difficult work. Because of the very close concentration, it's hard on the eyes, it's hard on the, you know, other small muscles that have to do with the melting. So I just want to mention that as sort of this unexpected side effect of the work, you know, not only does the translation need to be carefully done, and quality controlled, and all of that, but then placing, placing it into a webpage, if it's a manual process, which for us it is, then there's that aspect as well. And it's super attention to detail, because you have to figure out exactly where it goes. And for me a language like French or Spanish is easier to place and figure out where it goes. Then something like Amalric, which to me is it's a new language to me. And when that, you know, I am much less familiar with the characters. And so that then requires super concentration in order to get it right, because of course, I do want to put it into the right place. Otherwise, what's the point in doing that? So to recap that sometimes it's very small phrases or sentences that need to be replaced, in order to make the update. And so I feel like that's, you know, that's a different way of thinking about translation. So not only is that is it oftentimes sensitive, it's these snippets that need to be translated. And, you know, from a cost perspective, and how do you handle the money? Thankfully, I didn't need to think about the money and our process. And if you do, you know, how do you do that we did have some texts that we would have needed to have translated externally. And so then that, you know, I think twice about what is that going to be like? And who all needs to approve it before it goes? And darn it, it might change tomorrow? So yeah, it's been an interesting time. We've learned quite a lot. Clark Hayes 22:22 Yeah, that's, it's very interesting. And a good good segue to the second topic, which is budget and costing. And it sounds like projects can vary from a couple of words in the sentence to 10,000 words for the entire website. On each of your in each of the organizations, you've probably handled resources, vendors technologies differently. What kind of challenges do you run into from a budgeting perspective? In getting the resources and tools that you need? And what are some of the approaches that you've taken to to handle that? Maybe Peggy we can we can start with you on that one? Peggy Liao 23:02 Yeah, the one of the challenges is always that people will say like, I don't have money for translation. And so we are, we don't have a centralized language access budget that other departments can pull from. So a really big component of my job is also to really persuade people and then encourage people to explore language access services, and then include that in their campaign. So be able to support them with data and then and give them enough time to budget for language access. And then really be able to leverage resources, existing resources, and really put translation or interpretation on there is a long journey and and we started to encourage people to make language access plans in 2017 and 18. And then last year, we did a an update to align our language access planning process with our budgeting cycle. So the budgeting cycle coming up for at 1.3 and four will be happening in June this year. And the reason to update the plan last year is for to reserve that long time for our language axes, program managers to be able to talk to their finance team and really to think about how much they should really budget for it. And and then they can work with me to plan for items and really navigate the whole process and resources. So it is you know, kind of a game of both changing minds and hearts and and also really to come back that reality that people are fighting for resources and and also because of the pandemic, like a lot of cities, counties, states, like in general, like people don't have money, and we need to find places to cut budget. So under that big climate, how we can keep moving forward and then talking about budget, ensuring that we do have resources down in the road for us to use, it's a big challenge, but we are doing that. And then another way is to really think about other ways to if we can reduce duplication, translation and really be able to think about translation and interpretation from a holistic perspective and point of view. It's also very important and something that we are doing Clark Hayes 26:15 in landscape wise as it is sent, you said, it's not a centralized budget. You're advising other agencies departments on how to structure their language access programs, is that an accurate? Peggy Liao 26:30 Um, it depends. So it is not centralized to a way that each department also don't have a centralized language access budget. So ideally, if each department, that's a Human Services Department, they don't rely on individual program managers to say how much money I need for translation department on the department level, people can say, okay, furlough managers, we have, like $20,000, this year that you can pull from, then that will be easier for program managers. And then another way to centralize it, you can be like, okay, the Office of Immigrant and Refugee Affairs, you have $2 million each year, and 30 departments can draw on your budget, that is another way to centralize it. But we still want to keep the balance of if people put in the dollar amount into their budget sheet, that is ownership. And they are given the responsibility to use the money wisely. And we also want them to be able to feel like that is their program. They're responsible to providing language service to people who they're engaging with. So a lot of ways that you can talk about centralization. But that's the thinking behind like how to balance the responsibility and ownership. Clark Hayes 28:08 Perfect and Santiago's. What does the landscape look like for your organization? Any particular challenges that you run into from a budgeting procurement perspective? Santiago Torres 28:19 Yeah, this is a very interesting because you know, when I got into this field, originally as a translator, is because I like works. And then increasingly has become from an operational side, such a big part of the work we do that crunching numbers. And as Peggy said, creating dashboards, data sets, metrics, visualizations, and this is basically what I do about half of the week, these days, especially now, that budget season started for us, and we have a new one. So everything is very timely. So I'm gonna give you a very rapid response of how the landscape looks like because new administration, new types of folks new types of people that we have to engage from or from, from early stage. So that's the first step that that we've been doing, because there's been consistent changes to procurement rules, procurement processes, new folks were trying to get to understand what these processes and rules are. Budgets are decreasing. And then the other is, of course, the challenge of trying to explain to folks, you know, the technicalities of our fields, like what a fuzzy match is, what a CAT tool is, why do we need all this stuff? I had to just recently talk to a contracting officer trying to explain minimum fees and what that means, you know, and so, so it's basically just a form of constant engagement. But the way that we've been approaching this, though, is not only trying to get the key stakeholders involved from early, early process, but using a lot of the legislative infrastructure that was built as well. And also like for folks to, to notice this are essential services, particularly with a pandemic of how much information We were able to produce and you know, we have data for that, and the turnaround times, and so forth, and also how much what the cost was and how we can become more cost effective in certain ways. You know, we're trying to push for more in house resources, because the ROI in our field is more significant than if we outsource it constantly, you know. And so it's we frame it as that scenario, right, but also a question of advancing equity, and language access. And to comply with the law. If we do not get this funds approved, if there's a gap in service, there's a problem. So problem compliance. And so the last thing the city wants his language access complaints to be filed, because there's a system for that. One of the city agencies for people filing which access complaints. The other is, again, as pay with things like how to present this. So of course, with a very accessible data, and the metrics into show the magnitude and the impact of the work. And in one thing, particularly if that again, because just because we inherited all of these RFPs, from from over a decade ago, that, that we provide clear justifications, and clear scopes of work, and the importance of clear wrecks that are needed. So we don't get just a mega vendor who's going to bid the lowest price. So we know it's not really realistic. Now market rate and up ended lingos market rate. And there's been a lot of things that are tied into all this conversation about quality and turnaround times, and so forth. They are, they're affected. So we have a baseline budget, but not everybody else does. So there's Department of Education. You know, they have massively translation interpretation unit, like 50, linguist on staff project manager, they really like an LSP. They're probably one of the largest sales piece here in New York City, actually, and the Department of Health has a small language unit language access unit and Services Unit. NYCHA, the New City Housing Authority. That's one too, but that's about it in us Our little shop here, which also kind of functions by what the local law has all these language access. rules that you have to translate into those languages that telephonic interpretation needs to be available. The language in the world doesn't factor budgets, it doesn't say that. Okay, we every agency, pretty much like Peggy was saying that every agency has this, like, bylaw this amount of money. So it's really how, and part of our planning and really of just like evangelizing language accident with justice to make sure that people budget this from the beginning. So while we have a small budget, two sources, a lot of agencies do not. So for us as part of the larger, something that's on my colleagues here on the policy side of language access trainings, language access, presentations, convening of language access coordinators, to see how they can advocate for these funds, how they can, what a source of funding avenues out there, how to create a proper RFP and solicitation requests, why the difference between in house and in house unit and outsourcing and so forth into strategy, different costs and procurement rules. And, you know, for us, it's an ongoing education process as well. We're constantly engaging with the mayor's office, for contracts with the MWBE office, you know, with different contracting agencies, just to ensure that we are on top of it, but it's challenging. You know, it's what we know, today might be different by the end of the week. So especially now, new administration, and so forth. Clark Hayes 33:29 Okay, perfect. And I know Suzanne, you said that you don't deal with budgeting on your end. Is that because they're internal resources are what what do you typically do when you have to structure your your translation programs? And your mutexes Yeah, Susanne Brunhart Wiggins 33:45 so I'm on the IT side of the house. And so I don't have, like, my work doesn't have a budget for that. So I'm relying on the, like, the customer departments do either have the money or right now we're in the beginnings, middle, whatever, of setting up a an in house language unit, and working out the specifics of how that's going to work. So yeah, so I don't come like my work doesn't come with its own translation budget. I'm relying on those other sources of money to provide the funding. The one thing that I will add to the budgeting discussion, is that, at least from my side, that the translations are not one and then you can check off and then they're done. It's an ongoing thing. So even if there's the upfront cost, you know, like, let's say, for the COVID work that I've been doing, that it was translated and, you know, that was paid for. Well, now the information changed. And in fact, it's going to keep changing and it changes quite a lot. So, you know, that comes back to how do we budget for how do we pay for these snippets of translation that are ongoing Things that I will also add are, they're not necessarily everyday language. So now you're also working in specialized language? And is there an additional cost for doing that, as opposed to more day to day wording that might be used? And, you know, in English, even we've had new words enter our vocabulary through this COVID time, right? Who knew when a contact free pickup was at the beginning of this, who knew about PCR tests? And you know, like all the, the ins and outs of all of this stuff? And what is the difference between an initial dose of a vaccine versus a booster shot versus an additional dose for somebody who is immunocompromised? And you know, what is immunocompromised? And what do you understand when you hear those phrases. So I feel like that is an additional wrinkle that we deal with on the government side, because it's not just that you're you need a translation into this language, it's that you need a translation. And it's medical terms, or you need a translation, and it's legal terms, or you need a translation. And now we're talking about building a road. And so, you know, if the same person has facility or the team has facility and all those subject areas, that's fantastic. And if they don't, how are we going to handle that? Clark Hayes 36:24 Perfect, and that's another another good segue into the next topic of quality. And the one thing I heard on even on the budgeting side is it's never really about the cost and trim value of it's more about the value of translation. So you can do it internally. You can do with external vendors you can do with individual translators, but it's how do you educate people about the best practices and then implement that to get the best value? And ultimately, it's about quality? So on the on the quality side, what are some of the things that you keep in mind kind of what would be some of the best practices and advice you would give somebody approaching translation for the first time to make sure that you have consistency and quality when dealing with all these different content types? And Peggy and Santiago and your case, different agencies, different divisions who do things independently? Peggy maybe we can start again with you. Peggy Liao 37:22 Yeah, well, I think about that quality and also cultural relevancy a lot when it comes to translation for our city and just in general, Washington State Pacific Northwest. And so that's the big decision that we made to go with working with local translators, instead of just giving our big contract to a vendor and then say, find your translators. And so, we now work with about 65 people on the team and they are individual contractors that we work with and know so with the transaction management system, it is easier for us to assign project to many people and then invoice accordingly to each departments who has who requested project. So basically, that flow is 30 departments can anyone with the department can submit their project, online and portal. And then we then assign the project after reviewing the English content for simplicity and readability, assign that to translators and reviewer for each language. And working with local translators really preserve the cultural relevancy and then really just the understanding of what is going on with his with the city, who the mayor is how we're talking about public safety and really like what what's going on pandemic wear mask or not, and if someone who don't live here, it is more likely to get the information wrong if they're just translating from their original country of language. And so, that is one thing that we decided to work in this workflow and also, because of we are getting request, so we, as the language access team can provide our recommendation in terms of the English content. So we always review for if the content is ready for translation, and we can communicate with To request her and say, Hey, we actually recommend instead of saying Open House for something, you just say you have a public hearing or you have a public meeting. So you don't translate it in a confusing way. Um, yeah. So those are the elements that we put in to ensure quality. And I'm sure we'll get into like transactional memory and terminology, but Suzanne, and Sandiaga talk about Clark Hayes 40:32 that. Suzanna, on your end? What kind of challenges do you face? What advice would you give to manage quality? Susanne Brunhart Wiggins 40:42 Yeah, from what I've seen, developing a glossary is super helpful. Also, so that from translation job to translation job, you have consistency, and the person doesn't have to read translate it or go through a lot of effort to try to remember what they did the last time. Or if it's different translators doing the work that they they come up with a consistent product. I'm thinking while I was listening to you, I was reminded that so my work is on the web, right? So I try to write for the web, in English, and, and I try to display the even the translated content in the same way. So for example, I don't want to, ideally, the translator gives me back the linked language. So I don't need to go back and figure out where exactly do I need to place the link. And this is one of the huge bonuses for me of the translation management software that is able to handle HTML. So in English, you wouldn't use click here as a link. And so that's what ideally I don't want to get that back in the other language, or, you know, I'm typically quite intentional in the English that I give to the translators to be linked. And, you know, this is where my not having facility and all the languages might be a minus, I don't know. You know, like, so I know, it would be good practice to link in, in English and in the languages that I am familiar in. And so I'm hoping that that's also good practice in the language that is being presented on the page. So for example, you know, to make it a meaningful link, so if a link were to stand on its own, again, thinking about equity, if somebody is coming with accessible technology in order to access the information, you know, you don't want it to say learn more, learn more, learn more, no matter what the language is in or click here, you want that to, to have meaning. And you want things to be bulleted, and you want things to have headings. And so I feel like the more like we there used to be a series of radio ads here in this area that said, an educated consumer is your best customer. And so I feel like that with the translators as well like, the more that we have that shared background in what is writing for the web, what is plain language, when it comes to the medium that we're working in, that we then all can lift up our work together, and have that be the most successful presentation of the information that we're trying to get out there. Because Peggy Liao 43:18 in the end, Susanne Brunhart Wiggins 43:19 people who are coming to our websites, whether it's on a phone, whether it's on some other device, especially in this time, you have limited capacity to understand, regardless of the language, right, because there's so much going on, there's so much so many pressures that we have on our thinking. So we want that to be as easy as as easy to consume as it possibly can be, regardless of the language and so yeah, so knowing about you know, even that writing for the web as a thing, I think is useful. Clark Hayes 43:55 In and Santiago, on your end, when it comes to quality, what do you think about what kind of advice would you give? Santiago Torres 44:00 Yeah, I'm in all translations. We all, we all receive feedback. For anybody. As we know, one of the challenges here is that a lot of bilingual folks, well, really well intentioned, you know, they provide a feedback without really following from the linguistic quality metrics, you know, Michigan's Miss translations, you know, and so forth. And this just recently received one way, it was just words were just circled in a scan translation that was done. And we're like, what is this like, you figure it out? We just don't like it. You know. And so, one of the, that our hoc nature while we've been working to centralize is still there. Because once we release a translation, it's, you know, people have it and there's gonna be feedback and there's so many languages too. So that and regional differences. So for instance, in Yiddish, For example, just Yiddish. Here in New York, what's spoken in central Brooklyn is very different than when I live in northern Brooklyn. So there's going to be difference there with Bangla. There's in parts of Brooklyn. It's a particular part of Bangladesh, then, in Jackson Heights, Queens, so and we received that feedback and oldest inactive, just Susan mentioned, in all these languages, aside from Spanish, and some French, I really have no, it's hard for me to vet that. So what we did, and what we advise agencies and other peers and colleagues, once when possible, to try to centralize and streamline the feedback loop, and to see what any patterns and the types of feedback and one of them is also to provide, if folks want to be, I mean, not linguists, you know, just in the case of a linguist, which is always the case that somebody's going to give you feedback, or at least provide some simple metrics as to the different categories. And they'd like sweet colonists, we'd like penalties for a severity of mistranslations, and omissions and so forth is a grammar issue, or syntax, or style, and so forth. And then we can break that into categories. And we can assess just the gravity of of the issue of quality. But prior to the nuts, but before that, one thing that we do to start with, and you know, we talked about glossaries. And so for Spanish, we have a lot of control, we're really the gatekeepers for the content here for the mayor's office, at least. Because we already have term bases have a ton of terms that we've been accumulating throughout the years. Of course, as I mentioned, sector terminology keeps on changing. I didn't have to put into that term base, how to do a nasal swab for home test, you know, then we decided like, alright, if we do this, my colleague who does take the first crack at the translation, she's Peruvian, Ecuadorian, so like, how do you say this in Peru, I say like this in Ecuador, but we live in New York, alright, let's see if we can find a thing. So a great resource has been at least to do some agreements on the religious community, ethnic media, there's a lot of other human New York, so it's very good. And so we can reach out to some folks to help provide some input. But again, just something when working with the vendors, at least, when, as I mentioned earlier, what we did during this auditing phase, is okay, we were able to change vendors and create this new solicitation request and put quality and you know, even liquidated damages language here for any type of cases, but also to give them you know, more parameters here. So we did create some style guides, including some glossary, so at least with the most common terms that we like, the name of our agency, for instance, the name of other a few other agencies very difficult to coordinate, we try to do it through convenience and so forth. But against very difficult in government. But then, so at least we're like, okay, if we can get in the local 30 languages, so the 10 languages, maybe by the city, like, let's get at least the agency names, let's get something that our programs are sharing. At that time, particularly when I started with, like, TPS, Temporary Protected Status, you know, like DACA, and all these programs on the immigration level, but now increasingly has been on the health, healthcare and so forth. So we're trying to update those groceries, our again, our Spanish teams, organically have been able to give us consistency and accuracy. But we do pass those along to the vendors that like to make sure you follow these glossaries, the style guide, again, with the vendor, the good one, they actually helped us create some of these glossaries and sometimes give them the distributed create them pro bono. But again, they're a little bit of a different type of vendor. And another thing that we've done is just we've rewarded the RFP language, so that not only us here, Maria, we can get the best vendors, but really also the agencies because again, one thing that is happening in New York, I'm sure maybe it's the same with we're here with the archivist, the panel is that a lot of folks who are language access coordinators for other agencies, they don't come from the industry. So we need to forgive them share that information along. How do you evaluate an RFP? How do you evaluate quality? What does that mean? You know, how do you do terminology management? What are capsules? So that's like another part of our portfolio of trying to inform a different stakeholders. But I mean, it's, even with the most sophisticated tools, we have, say, like, are the most streamlined and efficient workstream that we have as a website. There's always still, you know, quality issues, people come back to us and that is, you know, a vetted process with linguists have been working with the same linguists work on the same projects across materials. So it's consistent, you know, we tried to integrate it teams were printed work and so forth. But it still happens, you know, so it comes with the territory. Clark Hayes 49:52 Perfect, thanks. Thanks, Santiago. We have about 10 minutes left. So the last last question that I'd like to cover is just open to you guys. And what one piece of advice would you give somebody who's approaching this for the first time whether they're in the public sector or private sector? What's the one piece of advice you'd give someone looking at translation for the first time? Santiago, if you want to start. Santiago Torres 50:20 Yeah, I mean, so what we do, and we actually, again, we're doing this again, because we have to provide a series of trainings to new folks, which is during this administration is we framed this as a question of equity and language justice, and of course compliance. But first piece of advice is we tell them to think multilingual, like, Please don't think multilingual from the start to be considerate and respectful of our limited English proficiency New Yorkers communities to plan that understand the language axis and look coefficients to collaborative effort to creative effort. And that in some cases requires time. And this requires subject matter experts, linguists proofreaders types that are so forth. And so to plan properly, to budget accordingly, and, and to coordinate and naturally, that we're here always for assistance with that. But the main, the main message is think multilingual from the side. Think about our constituents. Clark Hayes 51:25 Perfect, good advice. And Peggy, on your end. Peggy Liao 51:32 Yeah. I, so a lot of cities and counties will come to me and say, Hey, can you offer a piece of advice? How do I start? I will always say, just hire someone internally, whose job would be thinking about language axes, language, justice, transaction interpretation, day and night, it is really helpful to have a designated person team to keep pushing for cultural change and then keep including tools and resources into the workflow. And, and then we can expand that to like, really encouraging people to think, in the multilingual way. Yeah, so that is, and that does not only apply in the government setting, I think a lot of companies and organization can think about if they do work with people who don't speak English or speak prefer languages other than English. It is really helpful to have someone who's on your team can advocate for users of different languages. Clark Hayes 53:04 Perfect. And Suzanne, what one piece of advice might you give? Susanne Brunhart Wiggins 53:11 This week, my advice, having just move at the beat being at the beginning of a new project, is to really make the this multilingual view in from the very beginning. So when you think about a new website, think about so what languages is this going to be in? You know, don't assume it's going to be in any one language, but just ask that open ended question what languages is going to be in? Who's going to be responsible for those who's managing the translation process? Because that is a process in itself. You know, they're building a new website is a process. And so this sub process is the multilingual aspect of it. So just like when you're doing good user experience, or making a good user interface, you want that to be thought about from the very beginning and not be this afterthought at the end, keep lifting up this need for language access, and how you're going to handle it and incorporate it at every step. So that it's not a surprise at the end. Or it's not this, oh, my goodness, this thing that we need to do at the end. But you know, how is this going to be handled? How is it going to be displayed? How is it going to be presented? How are you managing the translations? So that it's just a natural part of how you do this work? Perfect. Peggy Liao 54:36 Another sentiment that I've observed from people who don't speak other languages, but English, is that they, like people will feel like Oh, my God, I don't know about translation and interpretation. And I don't want to do it wrong. Therefore, I don't do it. And I think they People make mistake and people make language access complaints. Anyway sometimes and so it is better to have a open minded and then and then just just put your resources out but also have that build up that expectation that you are doing wrong, but you will correct it and then that building trust and relationship with community members is way more important than being afraid of doing something wrong. Clark Hayes 55:38 Perfect. That's all really good advice. I want to thank Peggy, Suzanne Santiago for for joining us. And for everybody who, who watched the video today, we're gonna post it on LinkedIn. We'll have the ability to put comments in the comment field and then we'll get those to Peggy Santiago and Suzanne to answer if anybody has any, any comments, but just want to thank everybody, again, really informative and valuable discussion. I hope everybody as well has a great rest of the week. Thank you