Tech Law Talks

AI-driven autonomous ships raise legal questions, and shipowners need to understand autonomous systems’ limitations and potential risks. Reed Smith partners Susan Riitala and Thor Maalouf discuss new kinds of liability for owners of autonomous ships, questions that may occur during transfer of assets, and new opportunities for investors.

----more----

Transcript:

Intro: Hello and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting edge issues on technology, data and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day. 

Susan: Welcome to Tech Law Talks and our new series on AI. Over the coming months, we'll explore the key challenges and opportunities within the rapidly evolving AI landscape. And today we will focus on AI in shipping. My name is Susan Riitala. I'm a partner in the asset finance team of the transportation group here in the London office of Reed Smith. 

Thor: Hello, I'm Thor Maalouf. I'm also a partner in the transportation group at Reed Smith, focusing on disputes. 

Susan: So when we think about how AI might be relevant to shipping, One immediate thing that springs to mind is the development of marine autonomous vessels. So, Thor, please can you explain to everyone exactly what autonomous vessels are? 

Thor: Sure. So, according to the International Maritime Organization, the IMO, a maritime autonomous surface ship or MASS is defined as a ship which, to a varying degree, can operate independently of human interaction. Now, that can include using technology to carry out various ship-related functions like navigation, propulsion, steering, and control of machinery, which can include using AI. In terms of real-world developments, at this year's meeting of the IMO's working group on autonomous vessels, which happened last month in June, scientists from the Korean Research Institute outlined their work on the development and testing of intelligent navigation systems for autonomous vessels using AI. That system was called NEEMO. It's undergone simulated and virtual testing, as well as inland water model tests, and it's now being installed on a ship with a view to being tested at sea this summer. Participants in that conference also saw simulated demonstrations from other Korean companies like the familiar Samsung Heavy Industries and Hyundai of systems that they're trialing for autonomous ships, which include autonomous navigation systems using a combination of AI, satellite technology and cameras. And crewless coastal cargo ships are already operating in Norway, and a crewless passenger ferry is already being used in Japan. Now, fundamentally, autonomous devices learn from their surroundings, and they complete tasks without continuous human input. So, this can include simplifying automated tasks on a vessel, or a vessel that can conduct its entire voyage without any human interaction. Now, the IMO has worked on categorizing a spectrum of autonomy using different degrees and levels of automation. So the lowest level still involves some human navigation and operation, and the highest level does not. So for example, the IMO has a degree Degree 1 of autonomy, a ship with just some automated processes and decision support, where there are seafarers on board to operate and control shipboard systems and functions. But there are some operations which can be automated at times and be unsupervised. Now, as that moves up through the degrees, we get to, for example, Degree 3, where you have a remotely controlled ship without seafarers on board the ship. The ship will be controlled and operated from a remote location. All the way up to degree four, the highest level of automation, where you have a fully autonomous ship, where the operating systems of the ship are able to make their own decisions and determine their own actions without human interaction. action. 

Susan: Okay, so it seems like from what you said, there are potentially a number of legal challenges that could arise from the increased use of autonomy in shipping. So for example, how might the concept of seaworthiness apply to autonomous vessels, especially ones where you have no crew on board? 

Thor: Yeah, that's an interesting question. So the requirement for seaworthiness is generally met when a vessel's properly constructed, prepared, manned and equipped for the voyage that's intended. Now, in the case of autonomous vessels, they're not going to be able to. The kind of query turns to how a shipowner can actually warrant that a vessel is properly manned for the intended voyage where some systems are automated. What standard of autonomous or AI-assisted watchkeeping setup could be sufficient to qualify as having excised due diligence? A consideration is of course whether responsibility for seaworthiness could actually be shifted from the shipowner to the manufacturer of the automated functions or or the programmer of the software of the automated functions on board the vessel as you're aware the concept of seaworthiness is one of many warranties that's regularly incorporated in contracts for the use of ships and for carriage of cargo. And a ship owner can be liable for the damage that results if there's an incident before which the ship owner has failed to exercise due diligence to make the ship seaworthy. And this, in English law, is judged by the standard of what level of diligence would be reasonable for a reasonably prudent ship owner. That's true even if there has been a subsequent nautical fault on board. But how much oversight and knowledge of workings of an autonomous or AI-driven system could a prudent ship owner actually have? I mean, are they expected to be a software or AI expert? Under the existing English law on unseaworthiness, a shipowner or a carrier might not be responsible for faults made by an independent contractor before the ship came into their possession or before it came into their orbit. So potentially faults made during the shipbuilding process. So to what extent could any faults in an AI or autonomous system be treated in that way? Perhaps a ship owner or carrier could claim a defect in an autonomous system came about before the vessel came into their orbit and therefore they're potentially not responsible for subsequent unseaworthiness or incidents that result. There's also typically an exception to a ship owner's liability for navigational faults on board the vessel if that vessel has passed a seaworthiness test. But if certain crew and management functions have been replaced by autonomous AI systems on board, how could we assess whether there's or not there has actually been a navigational fault for which the owners might escape liability or pre-existing issue of unseaworthiness, so a pre-existing hardware or software glitch? This opens up a whole new line of inquiry as to at what might have happened behind the software code or the protocols of the autonomous system on board and the legal issues of responsibility of the ship owner and the subsequent applicable liability for any incidents which might have been caused by unseaworthiness are going to involve a significant legal inquiry and in new areas where it comes to autonomous vessels. 

Susan: Sounds very interesting. And I guess that makes me think of, I guess, a wider issue that crewing is only part of, which would be standards and regulations relating to autonomous vessels. And obviously, as a finance lawyer, that would be something my clients will be particularly interested in, in terms of what standards are there in place so far for autonomous vessels and what regulation can we expect in the future? 

Thor: Sure. Well, the answer is at the moment, there's not very much. So as I've mentioned already, the IMO has established a working group on autonomous vessels. And the aim of that IMO working group is to adopt a non-mandatory goal-based code for autonomous vessels, the MASS code, which will aim to be in place by 2025. But like I said, that will be non-mandatory, and that will then form the basis for what's intended to be a mandatory MASS Code, which is expected to come into force on the 1st of January 2028. Now, the MASS Code working group last met in May of this year. And it reports on a number of recommendations for inclusion in the initial voluntary MASS Code. Interestingly, one of those recommendations was for all autonomous vessels, so even the fully autonomous degree four vessels, to have a human being, a person in charge designated as the master even if that person is remote at all times so that may rule out a fully autonomous non-supervised vessel from being compliant with the code. So mandatory standards still very much under develop in development and not currently in force until 2028 at the moment that doesn't mean to say there won't be national regulations or flag regulations covering those vessels before then. 

Susan: Right. And then I guess another area there would be insurance. I mean, what happens if something happens to a vessel? I mean, I'm looking at it from a financial perspective, of course, but obviously for ship owners as well, insurance will be the key source of recovery. So what kinds of insurance products would already be available for autonomous vessels? 

Thor: Well, good to know that some of the insurers are already offering products covering autonomous vessels. So just having Googled what's available the other day, I bumped into Ship Owners Club, which holds entries for between 50 and 80 autonomous vessels under their All Risks P&I cover. And it seems that Guard is also providing hull and machinery and P&I cover for autonomous vessels. And I can see that their industry is definitely taking steps to get to grips with cover for autonomous vessels. So hull and P&I cover is definitely out there. So we've covered some of the legal challenges and insurance and what autonomous vessels are. I wonder, Susan, what other more specific challenges people interested in financing autonomous vessels might face? 

Susan: Sure. Yeah. So, I mean, I guess I'll preface that by saying that I'm an asset finance lawyer. So instinctively, when I think about financing autonomous vessels, I'm thinking about the assets itself. So either financing the construction or the acquisitions of of the vessel. But in terms of autonomous vessels in particular, there are boundless investment opportunities beyond just the vessel itself, you know, on the financing, some of the research and development, some of the corporate finance of the companies designing and building those vessels, and the technology used to operate them. So there's, I imagine, a vast opportunity here for an investor who's keen to get involved. From a commercial perspective, autonomous vessels are pretty new. They're pretty untested. Obviously, you've talked a lot about the fact that a lot of the regulation isn't really completely there yet. There's a lot of development still to come. So it takes quite a brave investor to put funding into it. And so far, at least, the return on investment is a bit uncertain. It's not like investing in a tanker or a bulk carrier where you've got a known market. Everyone knows what the problems are. Everyone knows what the risks are, how to mitigate them. So in a lot of ways, this is all still very, very new, both for the owners and for the finances. But investors are very interested in sustainability solutions. They're interested in what the next big thing is. So I imagine that the autonomous ships are quite likely to appeal with potentially better safety records, being more sustainable. That in turn would then make the asset better value for the investors and less likely to result in insurance claims or reputational damage resulting from incidents and that sort of thing. From a legal perspective, it doesn't immediately seem that there would be a huge difference in taking a mortgage over an autonomous ship versus a manned one. But then it becomes a bit more complicated if we start to think about enforcing that mortgage. So in the traditional way to enforce a mortgage, the mortgagee will arrest the vessel in a suitable port. Depending on where the vessel is, the lender may need to instruct the borrower or the manager to sail the vessel to a suitable port. And if the borrower fails to do this, the lender can become a mortgagee in possession, take over the ship, sail it into a friendly port and apply for traditional sale. But how are you going to do that if you can't just go on board and say to the master, hey, I've arrested this ship, I'm going to take over now. And thinking about, for example, the degree three vessels where you'd have a remote operator redirecting the ship, what happens? Presumably the mortgagee would have to go to them and say we'd like you to redirect this vessel what if they refuse can the lender take over can they override the autonomous system or the remote operation would they have to. Would there be cybersecurity issues, issues with password and access and things like that? I mean, these are all kind of big questions at the moment that no one's tried to do this yet. So it isn't really clear how all of this would fit in with the existing law on the rights of a mortgagee in possession, which is a very well-tested legal concept, but it does assume physical control of the ship, which is not as obvious in an autonomous scenario as it would otherwise be. And a conducted issue to that would be, what I already mentioned, is kind of the absence of a clear market, and this would be relevant in the context of a judicial sale. So at least at the outset, valuing autonomous vessels could be a bit difficult. And until there's a clearly defined secondhand market, it might be difficult to lend us to determine whether it's even worth enforcing in terms of the potential return they would get, because it's difficult to analyze how much you might be able to get for the vessel. Not aware of any cases where someone has tried to do this. So the existing law will definitely need to develop and it's going to be very interesting times as we navigate these changes in the market in relation to autonomous vessels. 

Thor: Yeah, I can see that autonomy definitely throws up a whole bunch of issues for financing. 

Susan: Definitely. I mean, at the moment, we don't entirely know all the answers, but we're definitely looking forward to finding out. 

Thor: Right. 

Susan: Thank you so much for joining us for our AI podcast today. 

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's emerging technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.

In this episode, we explore the intersection of artificial intelligence and German labor law. Labor and employment lawyers Judith Becker and Elisa Saier discuss key German employment laws that must be kept in mind when using AI in the workplace; employer liability for AI-driven decisions and actions; the potential elimination of jobs in certain professions by AI and the role of German courts; and best practices for ensuring fairness and transparency when AI has been used in hiring, termination and other significant personnel actions.

----more----

Transcript: 

Intro: Hello and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with everyday.

Judith: Hello, everyone. Welcome to Tech Law Talks and to our new series on AI. Over the coming months, we'll explore the key challenges and opportunities within the rapidly evolving AI landscape. Today, we will focus on AI in the workplaces in Germany. We would like to walk you through the employment-level landscape in Germany and would also like to give you a brief outlook on what's yet to come, looking at the recently adopted EU regulation on artificial intelligence, the so-called European Union AI Act. My name is Judith Becker. I'm a counsel in the Labor and Employment Group at Reed Smith. I'm based at the Reed Smith office in Munich, and I'm here with my colleague Elisa Saier. Elisa is an associate in the Labor and Employment Law Group, and she's also based in the Reed Smith office in Munich. So, Elisa, we are both working closely with the legal and HR departments of our clients. Where do you already come across AI in employments in Germany and what kind of use can you imagine in the future?

Elisa: Thank you, Judith. I am happy to provide a brief overview of where AI is already being used in working life and in employment law practice. The use of AI in employment law practice is not only increasing worldwide, but certainly also in Germany. For example, workforce planning and recruiting can be supported by AI. Therefore, already a pretty large number of AI tools does exist for recruiting, for example, in the job description and advertisement, the actual search and screening of applicants, as well as in the interview process, the selection and hiring of the right match, and finally the onboarding process. AI-powered recruiting platforms can make the process of finding and hiring talents more efficient, objective, and data-driven. These platforms use advanced algorithms to quickly scan CVs and applications and automatically pre-select applicants based on criteria such as experience, skills, and educational background. This does not only save time, but also improves the accuracy of the match between candidates and vacancies. In the area of employee evaluation, artificial intelligence offers the opportunity to continually analyze performance data and evaluate them. This enables managers to make well-founded decisions about promotions, salary adjustments, and further training requirements. AI is also used in the field of employee compensation. By analyzing large amounts of data, AI can identify current market trends and industry-specific salary benchmarks. This enables companies to adjust their salaries to the market faster and more accurately than with traditional methods. When terminating employment relationships, AI can be used with the social selection process, the calculation of severance payments, and drafting of warnings and termination letters. Finally, AI can support compliance processes, for example, in the investigation of whistleblowing reports received via Ethic Hotline. Overall, it is fair to say that AI has arrived in practice in the German workplace. This certainly raises questions about the legal framework for the use of AI in the employment context. Judith, could you perhaps explain which legal requirements employers need to consider if they want to use AI in the context of employment?

Judith: Yes, thank you, Elisa. Sure. The German legislature has so far hardly provided any AI-specific regulations in the context of employment. AI has only been mentioned in a few isolated instances in German employment laws. However, this does not mean that employers in Germany are in a legal vacuum when they use AI. There are, of course, general and not AI-specific employment laws and employment law principles that apply in the context of using AI in the workplace. In the next few minutes, we would like to give you an overview on the most relevant of these employment laws that German-based employers should have in their mind when they use AI. Now, I would like to start with the General Equal Treatment Act, the so-called AGG. Employers in Germany should definitely have that act in mind as it applies and it can also be violated even if AI is interposed for certain actions. According to this act, discrimination against job applicants and employees during their employments on the grounds of race or ethnic origin, gender, religion or belief, disability, age or sexual orientation is generally speaking prohibited. Although AI is typically regarded as something which is being objective, AI can also have biases and as a result the use of AI can also lead to discriminatory decisions. This may occur when, for example, training data the AI is trained with itself is based on human biases, and also if the AI is programmed in a way that is discriminatory. Currently, for example, as Elisa explained in the beginning, AI is very often used to optimize the application proceedings and when a biased AI is used here, for example, for selecting or for rejecting applicants, this can lead to violations of the General Equal Treatment Act. And since AI is not a legal subject itself, this discrimination would be attributable to the employer that is using the AI. And the result would then be, in the event of a breach of the Act, that the employer is exposed to claims for damages and compensation payments. And in this context, it is important to know that under the German Equal Treatment Act, the employee only has to demonstrate that there are indications that suggest the discrimination. So if the employee is able to do so, then the burden of proof shifts to the employer and the employer must then prove that there was in fact no such discrimination. And when an employer uses AI due to the complexity and the technical complexity that is involved, that can be quite challenging. In this regard, we think that a human control of the AI system is key and should be maintained. As we heard from Elisa in the beginning, AI is not only used in the hiring process, but also in the course of the employment. One question that came up here is whether AI can function as a superior itself and whether AI can give work instructions to employees. So the initial answer here is yes. German law does not stipulate any obligations that work instructions have to be given by a human being. Therefore, just as it is possible to delegate the right to give instructions to a manager or to another superior, it is also possible to enable an AI system to give instructions to the employees. In this context, it is important to recall, however, that the instructions are, of course, again attributable to the employer. And if the AI instructs in a way that is, for example, outside of the reasonable discretion or gives instructions which are outside of the employee's contract, then this instruction would, of course, be unlawful and that would be attributable to the employer as well. One aspect that I would like to point out here is that if an AI system would lead to a decision towards the employee that has legal effects and impacts the employee in a very significant way, then such decisions may not be made exclusively by an AI. This is because of a principle that is to be found in the data protection laws, and Elisa will explain on this in greater detail. Another aspect of AI in the course of employment is whether employers can instruct their employees to use AI. Again, here the answer is yes. This is a part of the employer's right to give instructions, and this right covers not only if employees, should use AI at all or if they are prohibited to use it. It also covers what kind of AI can be used and to avoid any misunderstandings and to provide for clarity here, we advise that employees should have a clear AI policy in place so that the employees know what the expectations are. And what they are allowed to do and what they are not allowed to do. And in this context, we think it is also very important to address confidentiality issues and also IP aspects, in particular, if publicly accessible AI is used, such as chat GPT.

Elisa: Yes, that's true, Judith. I agree with everything you said. In connection with the employer's right to issue instructions, the question also arises as to the extent to which employees may use AI to perform their work. The principle here is that if the employer provides its employees with a specific AI application, they are allowed to use it accordingly. Otherwise, however, things can get more complicated. This is because under German law, employees are generally required to carry out their work personally. This means that they are generally not allowed to have other persons to do their work in their place. The key factor is likely to be whether the AI application is used to support the employee in performing a task or whether the AI application performs the task alone. The scope of the use of AI is certainly relevant here as well. If employees limit themselves to give instructions to the AI application for a work task and simply copy the result, this can be an indication for a breach of the personal work performance. However, if employees ensure that they perform a significant part of the work themselves, the use of AI should not constitute a breach of duty. Employers are also free to expressly prohibit the use of artificial intelligence. It is also possible for employers to set binding requirements as to which tasks employees may use AI for and what they must observe when doing so. In the event of violations, employers can then, depending on the severity of the violation, take action by issuing a warning or giving termination. Even without an express prohibition from the employer, employees are not permitted to use artificial intelligence or must at least inform the employer about the use of AI in order not to violate the obligation arising from the employment contract. Data protection law is particularly important here. For data protection reasons, employees may not be allowed to enter protected personal data in an AI dialog box. This is mainly due to the fact that some AI systems are hosted on servers with lower data protection standards than in the EU. General data protection principles that apply in the context of employee data protection and the information rights of the employees concerned must also be observed when setting up and using AI. As a general rule, data is no longer required must be deleted and incorrect data must be corrected. If the purpose of the data is changed, the employee's concern must be informed too. In addition, the GDPR imposes special information requirement for automated decision making, in particular for profiling. In these cases, employers must inform the data subject of the existence of automated decision making and provide information on the scope and intended effects of such processing for the data subject. This can be a challenge for employers in practice, especially if they are using AI developed by other providers and the employer therefore does not have much knowledge about the function of the AI system. When using AI, companies should also observe the ban on automated individual decision-making in accordance with Article 22 GDPR. This regulation states that decisions that have legal consequences for the data subject or significantly affect them may not be based solely on the automated processing of personal data. Examples of this include selection decisions when recruiting applicants or giving notice of termination. The background to this is the protection of employees who should not be completely subject to a processing system in important matters. Accordingly, decisions on hiring, promotions, terminations, or warnings can generally not be made conclusively by an AI system, but are subject to a human decision-making process. Although it should be permissible to use AI to prepare such decisions, it should be ensured that a human is involved in the final decision-making process for this type of decisions. This human involvement should also be documented by the employer for evidence purposes.

Judith: Okay. I want to briefly take another look at termination of employment, but from a slightly different angle. Many employees may fear now that their job positions will be eliminated because AI will basically take the job- they will be replaced by AI. So we had a quick look at whether AI can be a reason for termination in Germany. Well, we haven't seen any specific case law on this, probably too early, but we think that the German labor courts would apply the general principles that they apply in redundancy scenarios. This means that the employer that uses AI would have to demonstrate that due to the use of AI, the job duties of the affected employee are eliminated in full and thus there's no need for this employment anymore. This can in practice be quite challenging and the German labor courts will fully review whether job duties have in fact been eliminated. The German labor courts, however, won't review whether the decision to use AI is reasonable or not. They will just review whether this decision is obviously arbitrary or, for example, discriminatory. But the decision itself is part of the entrepreneurial freedom and the courts won't assess whether it is a good decision or not. The courts would probably also apply all other general principles in redundancy scenarios with respect to, for example, offering suitable vacancies and with respect to social selection processes. We do not know yet whether courts would apply a stricter standard when it comes to training measures, for example, to get a position holder fit for the new workplace here. We think that the case law should be monitored and we would see how the courts would decide in such scenario. So, Elisa, let's have a look at a German specialty and let's have a look at those German-based employers who have a works council. Are there any specific legal implications here?

Elisa: Sure, Judith. The introduction and use of AI as a technical system in a company is generally subject to co-determination of the works council if there exists one in the respective company. In addition, the Works Council has information and consultation rights and must therefore be involved when using AI. Moreover, it is important to note that the Works Council's right to information already starts with the planning process, so that the Works Council must be informed at an early stage before the AI is actually implemented. The Works Council also generally has the right to consult experts during the implementation of AI. A different assessment with regard to the co-determination rights of the Works Council can arise when using external AI systems. This is because these are generally used by employees via their own account, to which the employer has no access. In such cases, it is not possible for the employer to monitor the performance or behavior of employees. This in turn means that no co-determination rights of the Works Council are triggered. If the employer wishes to use AI to implement selection guidelines, for example for recruitment or the transfer of employees, the consent of the Works Council is also required. In this regard, it should be noted, though, that co-determination rights only exist with regard to employees, not applicable. Last but not least, the implementation of AI could constitute a change in operation within the meaning of the German Works Constitution Act if the statutory conditions are met. The employer must in this case consult with the Works Council about the effects of the AI on the employees and, if applicable, conclude a so-called social plan. In addition to the existing legal requirements under German law, which we have just discussed, employers in Germany and in the European Union should also keep an eye on the recently published AI Act. Judith, what are the relevant provisions of the new EU regulation that employers will have to be aware of in the future?

Judith: Well, the AI Act offers material probably for its own episode, but let me at least briefly give you an overview on the Act. July 12th, the European AI Act that was adopted by the Council of the European Union in May was finally published. And this Act is considered as the world's first comprehensive law regulating AI. The Act applies to both providers of AI and deployers of AI systems. Employers will usually be considered as deployers in the meaning of the Act, unless they are really involved in the development of AI systems themselves. It is important to know that the Act does not stipulate a minimum size of the company for application, so that basically means that the AI Act applies to, in the employment law context, to all employers in the European Union. Just in brief, the centerpiece of this act is a classification of AI systems into different risk levels. And the AI Act then allocates different obligations and compliance requirements to the different risk tiers. So the Act takes risk level approach. The AI systems that are used in the HR departments, just as Elisa described in the beginning of our discussion, they will regularly be classified as so-called high-risk systems in accordance with the Act. And this applies, among others, to AI systems which are used in the course of recruitment, task assignment, performance evaluation, promotion, and termination of employment. And employers using such high-risk AI systems in the EU will face specific compliance requirements under the Act in the future. Besides these risk-specific obligations, there are also general obligations stipulated in the AI Act that apply regardless of the specific risk tier, and these are basically transparency and information obligations, and employers have to face these as well when using AI systems. Violations of the Act can result in severe fines, and although most of the obligations under the Act will only enter into force within two years' time, this means in the course of 2026, we think that it does make sense and would be prudent to deal with the Act at an early stage, so that all the AI systems which are in use and which are planned to be used in the future are implemented in a way that is compliant with the AI Act. Also, we think that works councils probably will demand corresponding information and will deal with the AI Act in greater detail and will ask for information and training measures. Well, having said all this, Elisa, what would you recommend to German-based employers? What measures should they take?

Elisa: Yeah, based on the legal situation just discussed, it is important to keep in mind that employers are free to decide whether AI should be used in the company or not. If the decision is made to implement and use AI, appropriate instructions for employers should be in place. This can be regulated by clear clauses on AI use in employment contracts, work instructions, or even in works agreement if a works council exists. According to our experience so far, it is common for employers in practice to at least have a list of AI systems that are permitted or prohibited in the company. However, beyond that, we advise to define certain framework for the use of AI in the company by means of an AI policy. This policy should contain clear requirements and minimum standards for the use of AI and reflect the core values of the individual company so that employees know what is expected of them and what is allowed and not allowed when using AI. In addition, before AI is implemented and also during its use, employees should receive appropriate training on how to use AI when performing their work. For example, employees should be advised not to enter any personal data into the AI system and not to commit any copyright violations. They should also be made aware that AI systems do not always deliver correct results, but often only something that sounds likely and plausible. So, every result generated with AI should be critically questioned and reviewed in detail. Due to the need for training, it can be assumed that the Works Council, if there exists one, will demand that training courses be offered by the employer. If such training is required, under German law, employers must generally feel the costs incurred. Beside the legal fact that the Works Council does generally have certain information and co-determination rights under German law, as just described. Involving the Works Council at an early stage is also recommended in order to determine the next steps and timetable for the implementation of AI. Moreover, the involvement of the Works Council generally serves to increase employees' acceptance of digitalization in the workplace. As AI is still a fairly new topic that is in flux and constantly evolving. We recommend that employers in Germany monitor future development in legislation and case law and keep an eye on future changes.

Judith: So thank you very much everyone for listening and we keep you posted for other episodes of this podcast.

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's emerging technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.

Reed Smith partners Howard Womersley Smith and Bryan Tan with AI Verify community manager Harish Pillay discuss why transparency and explain-ability in AI solutions are essential, especially for clients who will not accept a “black box” explanation. Subscribers to AI models claiming to be “open source” may be disappointed to learn the model had proprietary material mixed in, which might cause issues. The session describes a growing effort to learn how to track and understand the inputs used in AI systems training.

----more----

Transcript:

Intro: Hello and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day. 

Bryan: Welcome to Tech Law Talks and our new series on artificial intelligence. Over the coming months, we'll explore the key challenges and opportunities within the rapidly evolving AI landscape. My name is Bryan Tan and I'm a partner at Reed Smith Singapore. Today we will focus on AI and open source software. 

Howard: My name is Howard Womersley Smith. I'm a partner in the Emerging Technologies team of Reed Smith in London and New York. And I'm very pleased to be in this podcast today with Bryan and Harish. 

Bryan: Great. And so today we have with us Mr. Harish Pillay. And before we start, I'm going to just ask Harish to tell us a little bit, well, not really a little bit, because he's done a lot about himself and how he got here. 

Harish: Well, thanks, Bryan. Thanks, Howard. My name is Harish Pillay. I'm based here in Singapore, and I've been in the tech space for over 30 years. And I did a lot of things primarily in the open source world, both open source software, as well as in the hardware design and so on. So I've covered the spectrum. When I was way back in the graduate school, I did things in AI and chip design. That was in the late 1980s. And there was not much from an AI point of view that I could do then. It was the second winter for AI. But in the last few years, there was the resurgence in AI and the technologies and the opportunities that can happen with the newer ways of doing things with AI make a lot more sense. So now I'm part of an organization here in Singapore known as AI Verify Foundation. It is a non-profit open-source software foundation that was set up about a year ago to provide tools, software testing tools, to test AI solutions that people may be creating to understand whether those tools are fair, are unbiased, are transparent. There's about 11 criteria it tests against. So both traditional AI types of solutions as well as generative AI solutions. So these are the two open source projects that are globally available for anyone to participate in. So that's currently what I'm doing. 

Bryan: Wow, that's really fascinating. Would you say, Harish, that kind of your experience over the, I guess, the three decades with the open source movement, with the whole Linux user groups, has that kind of culminated in this place where now there's an opportunity to kind of shape the development of AI in an open-source context? 

Harish: I think we need to put some parameters around it as well. The AI that we talk about today could never have happened if it's not for open-source tools. That is plain and simple. So things like TensorFlow and all the tooling that goes around in trying to do the model building and so on and so forth could not have happened without open source tools and libraries, a Python library and a whole slew of other tools. If these were all dependent on non-open source solutions, we will still be talking about one fine day something is going to happen. So it's a given that that's the baseline. Now, what we need to do is to get this to the next level of understanding as to what does it mean when you say it's open source and artificial intelligence or open source AI, for that matter. Because now we have a different problem that we are trying to grapple with. The problem we're trying to grapple with is the definition of what is open-source AI. We understand open-source from a software point of view, from a hardware point of view. We understand that I have access to the code, I have access to the chip designs, and so on and so forth. No questions there. It's very clear to understand. But when you talk about generative AI as a specific instance of open-source AI, I can have access to the models. I can have access to the weights. I can do those kinds of stuff. But what was it that made those models become the models? Where were the data from? What's the data? What's the provenance of the data? Are these data openly available? Or are they hidden away somewhere? Understandably, we have a huge problem because in order to train the kind of models we're training today, it takes a significant amount of data and computing power to train the models. The average software developer does not have the resources to do that, like what we could do with a Linux environment or Apache or Firefox or anything like that. So there is this problem. So the question still comes back to is, what is open source AI? So the open source initiative, OSI, is now in the process of formulating what does it mean to have open source AI. The challenge we find today is that because of the success of open source in every sector of the industry, you find a lot of organizations now bending around and throwing around the label, our stuff is open source, our stuff is open source, when it is not. And they are conveniently using it as a means to gain attention and so on. No one is going to come and say, hey, do you have a proprietary tool? Adding that ship has sailed. It's not going to happen anymore. But the moment you say, oh, we have an open source fancy tool, oh, everybody wants to come and talk to you. But the way they craft that open source message is actually quite sadly disingenuous because they are putting restrictions on what you can actually do. It is contrary completely to what the open-source licensing says in open-source initiative. I'll pause there for a while because I threw a lot of stuff at you. 

Bryan: No, no, no. That's a lot to unpack here, right? And there's a term I learned last week, and it's called AI washing. And that's where people try to bandy the terms, throw it together. It ends up representing something it's not. But that's fascinating. I think you talked a little bit about being able to see what's behind the AI. And I think that's kind of part of those 11 criteria that you talked about. I think auditability, transparency would be kind of one of those things. I think we're beginning to go into some of the challenges, kind of pitfalls that we need to look out for. But I'm going to just put a pause on that and I'm going to ask Howard to jump in with some questions on his phone. I think he's got some interesting questions for you also. 

Howard: Yeah, thank you, Bryan. So, Harris, you spoke about the open source initiative, which we're very familiar with, and particularly the kind of guardrails that they're putting around what open source should be applied to AI systems. You've got a separate foundation. What's your view on where open source should feature in AI systems? 

Harish: It's exactly the same as what OSI says. We are making no difference because the moment you make a distinction, then you bifurcate or you completely fragment the entire industry. You need to have a single perspective and a perspective that everybody buys into. It is a hard sell currently because not everybody agrees to the various components inside there, but there is good reasoning for some of the challenges. But at the same time, if that conversation doesn't happen, we have a problem. But from AI Verify Foundation perspective, it is our code that we make. Our code, interestingly, it's not an AI tool. It is a testing tool. It is written purely to test AI solutions. And it's on an Apache license. This is a no-brainer type of licensing perspective. It's not an AI solution in and of itself. It's just taking an input, run through the test, and spit out an output, and Mr. Developer, take that and do what you want with it. 

Howard: Yeah, thank you for that. And what about your view on open source training data? I mean, that is really a bone of contention. 

Harish: That is really where the problem comes in because I think we do have some open source trading data, like the Common Crawl data and a whole slew of different components there. So as long as you stick to those that have been publicly available and you then train your models based on that, or you take models that were trained based on that, I think we don't have any contention or any issue at the end of the day. You do whatever you want with it. The challenge happens when you mix the trading data, whether it was originally Common Crawl or any of the, you know, creative license content, and you mix it with non-licensed or licensed under proprietary stuff with no permission, and you mix it up, then we have a problem. And this is actually an issue that we have to collectively come to an agreement as to how to handle it. Now, should it be done on a two-tier basis? Should it be done with different nuances behind it? This is still a discussion that is ongoing, constantly ongoing. And OSI is taking the mother load of the weight to make this happen. And it's not an easy conversation to have because there's many perspectives. 

Bryan: Yeah, thank you, for that. So, Harish, just coming back to some of the other challenges that we see, what kind of challenges do you foresee the continued development of open source with AI we'll see in the near future you've already said we've encountered some of them some of the the problems are really kind of in the sense a man-made because we're a lot of us rushing into it what kind of challenges do you see coming up the road soon. 

Harish: I think the, part of the the challenge you know it's an ongoing thing part of the challenge is not enough people understand this black box called the foundational model. They don't know how that thing actually works. Now, there is a lot of effort that is going into that space. Now, this is a man-made artifact. This piece of software that you put in something and you get something out or get this model to go and look at a bunch of files and then fine-tune against those files. And then you query the model, and then you get your answer back, a rag for that matter. It is a great way of doing it. Now, the challenge, again, goes back to because people are finding it hard to understand, how does this black box do what it does? Now, let's step back and say, okay, has physics and chemistry and anything in science solved some of these problems before? We do have some solutions that we think that make sense to look at. One of them is known as, well, it's called Computational Fluid Dynamics, CFD. CFD is used, for example, if you want to do a fluid analysis or flow analysis over the wing of an aircraft to see where the turbulences are. This is all well understood, mathematically sound. You can model it. You can do all kinds of stuff with it. You can do the same thing with cloud formation. You can do the same thing with water flow and laminar flow and so on and so forth. There's a lot of work that's already been done over decades. So the thinking now is, can we now take those same ideas that has been around for a long time and we have understood them and try and see if we can apply this into what happens in a foundational model. And one of the ideas that's being worked on is something called PINN, which stands for Physics Informed Neural Networks. So using physics, standard physics, to figure out how does this model actually work. Now, once you have those things working, then it becomes a lot more clearer. And I would hazard a guess that within the next 18 to 24 months, we'll have a far clearer understanding of what is it inside that black box that we call the foundational model. With all these known ways of solving problems that, you know, who knew we could figure out how water flows or how, who knew we could figure out how, you know, the air turbulence happens over a wing of a plane. We figured it out. We have the math behind it. So that's where I feel that we are solving some of these problems step by step. 

Bryan: And look, I take your point that we all need to try to understand this. And I think you're right. That is the biggest challenge that we all face. Again, when it's all coming thick and fast at you, that becomes a bigger challenge. Before I kind of go into my last question, Howard, any further questions for Harish? 

Howard: I think what Harish just came up with in terms of the explanation of how the models actually operate is really the killer question that everybody is poised with the work the type of work that I do is on the procurement of technology for financial sector clients and when they want to understand when procuring AI what the model does it they often receive the answer that it is a black box and not explainable which kind of defies the logic of what their experience is in terms of deterministic software you know if this then that you know ] find it very difficult to get their head around the answer being a black box box methodology and often ask you know what why can't you just reverse engineer the logic and plot a point back from the answer as a breadcrumb trail to the input? Have you got any views on that sort of question from our clients? 

Harish: Yeah, there's plenty of opportunities to do that kind of work. Not necessarily going back from a breadcrumb perspective, but using the example of the PINN, Physics Informed Neuro Networks. Not all of them can explain stuff today. We have to, no one, an organization and a CIO who is worth their weight in gold should ever agree to an AI solution that they cannot explain. If they cannot explain, you are asking for trouble. So that is a starting point. So don't go down the path just because your neighbor is doing that. That is being very silly from my perspective. So if we want to solve this problem, we have to collectively figure out what to do. So I give you another example of an organization called KWAAI.ai. They are a nonprofit based in California, and they are trying to build a personal AI solution. And it's all open source, 100%. And they are trying really, really hard to explain how is it that these things work. And so this is an open source project that people can participate in if they choose to and understand more and at some point some of these things will become available as model for any other solution to be tested against so so and then let me then come back to what the verify foundation does we have two sets of tools that we have created one is to create One is called AI Verified Toolkit. What it does is if you have your application you're developing that you claim is an AI solution, great. Now, what I want you to do is, Mr. Developer, put this as part of your tool chain, your CICD cycle. When you do that, what happens, you change some stuff in your code. Then you run this through this toolkit, and the toolkit will spit out a bunch of reports. Now, in the report, it will tell you whether it is biased, unbiased, is it fair, unfair, is it transparent, a whole bunch of things it spits out. Then you, Mr. Developer, make a call and say, oh, is that right or is that wrong? If it's wrong, we'll fix it before you actually deploy it. And so this is a cycle that has to go continuously. That is for traditional AI stuff. Now, you take the same idea in the traditional AI and you look at generative AI. So there's another project called Moonshot. That's the name of the project called Moonshot. It allows you to test large language models of your choosing with some inputs and what outputs come up with the models that you are testing against. Again, you do the same process. The important thing for people to understand and developers to understand, and especially businesses to understand is, as you rightly pointed out, Howard, the challenge we have, this is not deterministic outputs. These are all probabilistic outputs. So if I were to query a large language model, like AAM in London, by the time I ask the question at 10 a.m. in Singapore, it may give me a completely different answer. With the same prompt, exactly the same model, a different answer. Now, is the answer acceptable within your band of acceptance? If it is not acceptable, then you have a problem. That is one understanding. The other part of that understanding is, it suggests to me that I have to continuously test my output every single time for every single output throughout the life of the production of the system because it is probabilistic. And that's a problem. That's not easy. 

Howard: Great. Thank you, Harish. Very well explained. But it's good to hear that people are trying to address the problem and we're not just living in an inexplicable world. 

Harish: There's a lot of effort underway. There's a significant amount. MLCommons is another group of people. It's another open source project out of Europe who's doing that. AI Verified Foundation, that's what we are doing. We're working with them as well. And there's many other open source projects that are trying to address this real problem. Yeah so one of the outcomes hopefully that you know makes a lot of sense is at some point in time the tools that we have created maybe be multiple tools can be then used by some entity who is a certification authority so to speak takes the tool and says hey Mr. company a company b, we can test your ai solutions against these tools and once it is done you pass we give you a rubber stamp and say you have tested against it so that raises the confidence level from a consumer's perspective, oh, this organization has tested their tools against this toolkit and as more people start using it, the awareness of the tools being available becomes greater and greater. Then people can ask the question, oh, don't just provide me a solution to do X. Was this tested against this particular set of tools, a testing framework? If it's not, why not? That kind of stuff. 

Howard: And that reminds me of the Black Duck software that tests for the prevalence of open source in traditional software. 

Harish: Yeah, yeah. In some sense, that is a corollary to it, but it's slightly different. And the thing is, it is about how one is able to make sure that you... I mean, it's just like ISO 9000 certification. I can set up the standards. If I'm the standards entity, I cannot go and certify somebody else against my own standards. So somebody else must do it, right? Otherwise, it doesn't make sense. So likewise, from AI Verify Foundation perspective, we have created all these tools. Hopefully this becomes accepted as a standard and somebody else takes it and then goes and certifies people or whatever else that needs to be done from that point. 

Howard: Yeah and and we we do see standards a lot you know in the form of iso standards recovering almost like software development and cyber security again that also makes me think about certification which we're is seeing appear in European regulation. We saw it in the GDPR, but it never came into production as something that you certify your compliance with the GDPR. We have now seen it appear in the EU AI Act. And because of our experience of not seeing it appear in the GDPR, we're all questioning, you know, whether it will come to fruition in the AI Act or whether we have learned about the advantages of certification, and it will be focused on when the AI Act comes into force on the 1st of August. I think we have many years to understand the impact of the AI Act before certification will start to even make a small appearance. 

Harish: It's one thing to have legislative or regulated aspects of behavior. It's another one when you voluntarily do it on the basis of this makes sense. Because then there is less of hindrance or less of resistance to do it. It's just like ISO 9000, right? No one legislates it, but people still do it. Organizations still do it because it's their, oh yeah, we are an ISO 9035 organization, And so we have quality processes in place and so on and so forth, which is good for those that is important. That becomes a selling point. So likewise, I would love to see something that right now, ISO 42001, which is all the series of AI-related standards. I don't think any one of them has got anything that can be right now be certified yet. Doesn't mean it will never happen. So that could be another one, right? So again, the tools that AI Verified Foundation creates and Mel Korman creates and everybody feeds into it. Hopefully that makes sense. I'd rather see a voluntary take-up rather than a mandated regulatory one because things change. And it's much harder to change the rules than to do anything else. 

Howard: Well, I think that's a question in itself, but probably it will take us way over our time whether the market forces us to drive standardization. And we could probably have our own session on that, but it's a fascinating subject. Thank you, Harish. 

Bryan: Exactly I think standards and certifications are possibly the kind of the next thing to look out for for AI you know Harish you could be correct. But on that note last question from me Harish so, interestingly the term you use moonshot right and so personally for you what kind of moonshot wish would you have for open source and AI. Leave aside resources, yeah if you could choose what kind of development would you think would be the one that you would look out for, the one that excites you? 

Harish: I would rather that, for me, we need to go all the way back to the start from an AI training perspective, right? So the data. We have to start from the data, the provenance of the data. We need to make sure that that data is actually okay to be used. Now, instead of everybody going and doing their own thing, Can we have a pool where, you know, I tap into the resources and then I create my models based on the pool of well-known, well-identified data to train on. Then at least the outcome from that arrangement is we know the provenance of the data. We know how it was trained. We can see the model. model, and hopefully in that process, we also begin to understand how the model actually works with whichever physics related understanding that we can throw at it. And then people can start benefiting and using it in a coherent manner. Instead of what we have today, I mean, in a way, what we have today is called a Cambrian explosion, right? There are a billion experiments happening right now. And majority, 99.9% of it will fail at some point. And 0.1% needs to succeed. And I think we are getting to that point where there's a lot more failures happening rather than successes. And so my sense is that we need to have data that we can prove that it's okay to get and okay to use, and it is being replenished as and when needed. And then you go through the cycle. That's really my, you know, Mojoc perspective. 

Bryan: I think there's really a lot for us to unpack, to think about, but I think it's really been an interesting discussion from my perspective. I'm sure, Howard, you think the same. And I think with this, I want to thank you for coming online and joining us this afternoon in Singapore, this morning in Europe on this discussion. I think it's been really interesting from a perspective of somebody who's been in technology and interesting for the ReadSmith clients who are looking at this from a legal and technology perspective. And I just wanted to thank you for this. And I also wanted to thank the people who are tuning into this. Thank you for joining us on this podcast. Stay tuned to the other podcasts that the firm will be producing, and I do have a good day. 

Harish: Thank you. 

Howard: Thank you very much. 

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's Emerging Technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.

The rapid integration of AI and machine learning in the medical device industry offers exciting capabilities but also new forms of liability. Join us for an exciting podcast episode as we delve into the surge in AI-enabled medical devices. Product liability lawyers Mildred Segura, Jamie Lanphear and Christian Castile focus on AI-related issues likely to impact drug and device makers soon. They also give us a preview of how courts may determine liability when AI decision-making and other functions fail to get desired outcomes. Don't miss this opportunity to gain valuable insights into the future of health care.

----more----

Transcript: 

Intro: Hello and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day.

Mildred: Welcome to our new series on AI. Over the coming months, we'll explore the key challenges and opportunities within the rapidly evolving AI landscape. Today, myself, Mildred Segura,, partner here at Reed Smith in the Life Sciences Practice Group, along with my colleagues, Jamie Lanphear and Christian Castile, will be focusing on AI and its intersection with product liability within the life sciences space. And especially as we see more and more uses of AI in this space, we've been talking about there's a lot of activity going on with respect to the regulatory landscape as well as the legislative landscape and activity going on there, but not a lot of discussion about product liability and its implications for companies who are doing business in this space. So that's what prompted our desire and interest in putting together this podcast for you all. And with that, I'll have my colleagues briefly introduce themselves. Jamie, why don't you go ahead and start?

Jamie: Thanks, Mildred. I'm Jamie Lanphear. I am of counsel at Reed Smith based in Washington, D.C. I'm in the Life Sciences and Health Industry Group. I've spent the last 10 years defending manufacturers and product liability litigation, primarily in the medical device and pharma space. I think, like you said, this is just a really interesting topic. It's a new topic, and it's one that hasn't gotten a lot of attention. A lot of airtime, you know, you go to conferences these days and AI is sort of front and center in a lot of the presentations and webinars. And much of the discussion is around, you know, regulatory cyber security and privacy. And I think that, you know, in the coming years, we're going to start to see product liability litigation in the AI medical device space that we haven't seen before. Christian, did you want to go ahead and introduce yourself?

Christian: Yeah, thanks, Jamie. Thanks, Mildred. My name is Christian Castile. I am an associate at Reed Smith in the Philadelphia office. And much like Mildred and Jamie, my practice consists primarily working alongside medical device and pharmaceutical manufacturers in product liability lawsuits. And Jamie, I think what you mentioned is so on point. It feels like everybody's talking about AI right now. And to a certain extent, I think that can be intimidating, but we actually are at a really interesting vantage point opportunity to get in the ground on the ground floor of some of this technology and how it is going to shape the legal profession. And so, you know, as the technology advances, we're going to see new use cases popping up across industries and, of course, of interest to this group in particular is that healthcare space. So it's really exciting to be able to grapple this headfirst and the people who are sort of investing in this now are going to be able to just really be a leg up when it comes to evaluating their risk.

Mildred: So thanks, Jamie and Christian, for those introductions. As we said at the outset, you know, we're all product liability litigators and based on what we're seeing, AI product liability is the next wave of product liability litigation on the horizon for those in the life sciences space and we're thinking very deeply about these issues and working with clients on them because of what we see on the horizon and what we're already seeing in other spaces in terms of litigation and that's, you know, what we're We're here to discuss today because of the developments that we're seeing in product liability litigation in these other spaces and the significant impact of, you know, what that litigation may represent for those of us in the life sciences space. And to level set our discussion today, we thought it would be helpful to briefly describe, you know, the kind of AI-enabled, you know, med tech or medical devices that we're seeing currently out there on the market. And I know, Jamie, you and I were talking about this, you know, in preparation for today's podcast in terms of, you know, just talking about FDA-cleared devices. I mean, what are the metrics that we're seeing with respect to that and the types of AI-enabled technology?

Jamie: Sure. So, we've seen a huge uptick in the number of medical devices that are incorporating artificial intelligence and machine learning. There are currently around 900 of those devices on the market in the United States, and more than 150 of those were authorized by FDA just in the last year. So, definitely seeing a growing number, and we can expect to see a lot more in the years to come. The majority of these devices, about 75%, are in the field of radiology. So, So for example, we now have algorithms that can assist radiologists when they're reviewing a CT scan of a patient's chest and highlight potential nodules that the radiologist should review. We see similar technology being used to detect cancer. So there's algorithms that can identify cancerous nodules or lesions that may not even be visible to a radiologist because they are undetectable by the human eye. And then other areas where we're seeing these devices being used, then cardiology and neurology.

Mildred: And I would add to that, you know, we're also seeing it with respect to, you know, surgical robots, right? And even though we don't have fully autonomous surgical robots out there on the market, you know, we do have some forms of surgical robots. And I think it's just on the horizon that we'll start to see, you know, in the near future, perhaps these surgical robots using, you know, artificial intelligence driven algorithms. And so that just the thought of that, right, that we're moving in that direction, I think, makes this discussion so important. And not just sort of in the medical device arena, but also within the pharma space where you're seeing the use of artificial intelligence to speed up and improve clinical development, drug discovery, and other areas. So you can see where the risks lie just within that space alone in addition to medical devices. And Christian, I know that you've been looking at other areas as well. So I wanted to tell us a little bit about those.

Christian: Sure. Yeah, and very similar to sort of the medical device space, there is a lot of really exciting room for growth and opportunity in the pharmaceutical space, seeing more and more technologies coming out that are focusing on streamlining things like drug discovery, using machine learning models, for example, to assist with identification of which molecules are going to be the most optimal to use in either pharmaceutical products, but also in the development of identifying mechanisms of action for being able to explain some of the medicines and the disease states that we have now that we're not able to explain as well. And then looking even more broadly, right? So you have, of course, these very specific use cases tied to the pharmaceutical products that we're talking about. But even more broadly, you'll see companies who are integrating AI into things like manufacturing processes, for example, and really working on driving the efficiency of the business, both from the product development standpoint, but also from a product production standpoint as well. So lots of opportunity here to get involved in the AI space and lots of ways to sort of grapple with how to best integrate it into a business.

Mildred: And I think that brings us to the question of what is product liability? For those listeners who may not be as familiar with the law of product liability, just to level set here too, you know, typically we're talking about three common types of product liability claims, right? You have your design defect, manufacturing defect, and failure to warn claims. Those are the typical claims that we see. And each of these scenarios or claims is premised on a product that leaves a manufacturer's facility, you know, with the defect in place, either in the product or in the warning. And these theories fit neatly for products that remain unchanged from the moment they leave the manufacturer's facility, such as consumer goods that are sold at retail. But what about when you start incorporating AI, machine learning technologies into these types of. Devices that are going to be learning and adapting, what does that mean for these types of product liability claims? And what is the impact? How will the courts deal and address and assess these types of claims as they start to see these types of devices and claims being made related to these types of technologies? And I think the key question that will come up right, is in the context of a product liability suit, is this AI-related technology even a product, right? And historically, courts have viewed software as a service that is not subject to product liability causes of action. However, that approach may be evolving to reflect the most Most products, you know, today maybe contain software or are composed entirely of software. We're seeing some litigation and other spaces that Jamie will touch on in a little bit that are showing sort of a change in that trend that we had been seeing and now moving in a different direction, which is something that we want to talk about. So maybe, Jamie, why don't you share a little bit about sort of what we're seeing in connection with product liability claims in other spaces that may inform what happens in the life sciences space.

Jamie: Yeah. So there have been a few cases and decisions over the last few years that I think help inform what we can expect to see with respect to products liability claims in the life science space, particularly around devices that incorporate artificial intelligence and software. One of those cases is the social media products liability MDL out of the Northern District of California. There you have plaintiffs who have filed suit on behalf of minors alleging that operators of various social media platforms designed these platforms to intentionally addict children. And this has allegedly resulted in a number of mental health issues and the sexual exploitation of minors. Now, last year, the defendants filed a motion to dismiss, and there were a lot of issues addressed in that motion, a lot of arguments made. We don't have time to go through all of them. But the one I do want to talk about that is relevant to our discussion today is the defendants argument that their social media platforms are not products, they're services. And as such, they should not be subject to product liability claims. And that argument is really in line with the historical approach courts have taken towards software, meaning that software has generally been considered a service, not a product. So software developers have generally not been subject to product liability claims. And so that's what the defendants argued in their motion, You know, that they were providing a platform where users could come, create content, share ideas. They weren't over in a warehouse making a good, distributing it to the general public, etc. So the court did not agree. The court rejected the defendant's argument and refused to take what it called an all or nothing approach to evaluating whether the plaintiff's design defect claims could proceed. And so the court took a more nuanced approach and it looked at the specific functions of these platforms that the plaintiffs were alleging was defective and evaluated whether each was more akin to tangible personal property or to ideas and content. So, for example, one of the claims that the plaintiff made was that the platforms lacked adequate parental controls and age verification. And so the court looked at, you know, what is the purpose of parental controls and age verification and its access? The court said this has nothing to do with sharing ideas, but this is more like products that contain parental controls, such as a prescription medicine bottle. And the court went through this analysis for each of the other allegedly defective functions. And interesting for each, it concluded that the plaintiff's product liability claims could proceed. And so what I think is huge to take away from this particular decision is that, you know, the court really moved away from the traditional approach courts have taken towards software with respect to product liability. And I think this really opens the door for more courts to do the same, specifically to expand products liability law, strict products liability to various types of software and software functions, such that the developers of the software can potentially be held liable for the software that they're developing. And while there have been a few one-off cases over the years, mostly in state court, in which the court has found that products liability liability law does apply to software. Here we have a huge MDL with a significant number of plaintiffs in federal court. And I think that this case is going to have, or this decision at least, is going to have a huge impact on future litigation.

Mildred: And I think that that's all really helpful, Jamie, in terms of the way you put the court's analysis. And I think one important to highlight is that in this particular case, the plaintiffs brought their causes of action both in strict liability as well as negligence. And I think the reason that's important to us and why it's of concern that you're seeing these plaintiffs, typically they might bring these types of claims under a negligence standard, which involves a reasonable person standard, assessing was there a duty to warn. And the court did look at some of that, you know, was there a duty here to the plaintiffs, but also strict liability, which is the one that you don't typically see brought in the case of, you know, software applications being discussed. And so the fact that you're seeing plaintiffs moving in this direction, asserting the strict product liability claims, in addition to negligence, which is what you would typically see, I think, is what is worth paying attention to. And, you know, this decision was at the motion to dismiss stage. So it will be interesting to see how it unfolds as the case moves forward through discovery and ultimately summary judgment. And it's not the only case out there. There are some other cases as well that are grappling with these issues. But this particular case, as Jamie noted, that the analysis was very detailed, very nuanced in terms of how the court got to where it did, you know, and it did a very thoughtful analysis going through, is it a software or a product? Once it answered that question and moved to, as Jamie noted, analyzing each of the product claims that were being asserted, and with failure to warn, it didn't really dive into that because of the way it had been pleaded. But nevertheless, it's still a very important decision from our perspective. And that was within sort of, you know, this product liability context. We've seen other developments in the case law. Involving cases alleging design defect, not necessarily in the product liability context, but more so in the consumer protection space, if you will. Specifically, one case that we were talking about in preparation for this podcast involving certain types. What was it, Jamie, the specific technology at issue?

Jamie: Yeah, so the Roots case is extremely interesting. And although it's a consumer protection case, not a products case, I do think that it foreshadows the types of new theories that we can expect to see in products liability litigation involving devices that incorporate software, artificial intelligence, and machine learning. So Roots Community Center is a California state case in which a community health center filed suit against manufacturers. Developers, distributors, and sellers of pulse oximeters, which are those devices that measure the amount of oxygen in your blood. And so the plaintiffs are alleging that these devices do not properly measure oxygen levels in people with darker skin, that the level of skin pigmentation can and does affect the output that these devices are generating by overestimating the oxygen level for these individuals. Individuals and that by doing so, these individuals are thinking that they have more oxygen than they do and they appear healthier than they are and they may not seek or receive the appropriate care as a result. And the reason for this, according to plaintiffs, is that the developers of the software when they were developing this device did not take into account the impact that skin color could have, that they essentially drew from data sets that were primarily white. And as such, they got results that. Largely apply to white folks. And so this issue of bias, right, is not one that I've ever seen raised as a theory of defect in a products case. And again, this isn't a products case, but I do expect to see this theory, products cases involving medical devices that incorporate artificial intelligence. You know, the FDA has been very clear that bias and health equity are are at the forefront of their efforts to develop guidelines and procedures specific to artificial intelligence machine learning-enabled devices. Particularly given that the algorithms depend on the data being used to generate output. And if the data is not reflective of the population who will be using the device and inclusive of groups like women, people of color, etc., the outputs for these groups may not be accurate.

Mildred: And what about with respect to failure to warn? You know, we know as product liability litigators that, you know, one of our typical defenses to a failure to warn claim is the learned intermediary doctrine, right? Which means that a manufacturer's duty to warn runs to the physician. You know, you're supposed to provide adequate warnings to the physician to enable them to be able to discuss the risks and benefits of a given device or pharmaceutical or treatment to the patient. But what happens to that? And then that's in the case of prescription medical devices or prescription pharmaceuticals, right? But what happens when you start incorporating AI and these, you know, machine learning technologies into a. Whether it's a medical device or in the pharma space, what happens to that learned intermediary defense? I mean, are you seeing anything that would change your mind in terms of learned intermediary doctrine is here to stay, it's not really going to change, right? I think if you ask me, I would say that based on what we're seeing so far, whether it's within the social media context or even cases that we've seen in the life sciences space that may not be specific to AI or machine learning. I think the fact that we're not yet at the stage where the technology is fully autonomous, it's more assistive, we're augmenting what a physician is doing, that will still require that you have this learned intermediary between the patient and the manufacturer who can speak to, you know, perhaps this treatment, whether it's through a medical type device or a pharmaceutical, is using this technology. Here's what it will be doing for you. This is the way it will function, et cetera. But, you know, does that mean that the manufacturer will have to make sure that they're providing clear instructions to the physician? You know, I think the answer to that is yes. And that's something that the FDA, through its guidance that it's put out, is looking at and has spoken to, Jamie, to your point, right? Not only with respect to bias, they're also looking to ensure that to the extent these technologies are being incorporated, that, you know, instructions related to their use are adequate for the end user, whether that's, you know, in many cases, the physician. But it also does raise questions as as these technologies get more sophisticated, who will be liable, right? What happens when you do start to see a more fully autonomous system who might be making decisions that. The physician just doesn't have the capacity to unpack, for instance, or fully evaluate, right? Who's responsible then? And I think that may explain a little bit of the reticence on the part of physicians to adopt these technologies. And I think ultimately, it's all about transparency, having clear, adequate information where they feel comfortable not only using the technology, right? But who ultimately will be responsible if, you know, God forbid, there's something that goes. Undetected, or the technology is telling the doctor to do something and the doctor overrides it, situations like that. And so I don't know if you all have any additional thoughts on that.

Christian: It's very interesting, right, this learned intermediary concept, I think, particularly because as we see this technology grow, we're going to see the bounds of this doctrine get stretched out a little bit. And to your point, Mildred, transparency here is going to be important, not only with respect to who is making those decisions, but also with respect to how those decisions are being made. So when you're talking about things like, how is the AI working and how are the algorithms that are underlying this technology coming to the conclusions that they are, it's going to be really important in this warning context that what's being discussed, how the AI is helping integrate into these products that everybody involved is able to understand specifically what that means. What is the AI doing? How is it doing it? And how does that translate to the medical service or the benefit that this product or pharmaceutical is providing? And that's all interesting and going to be, I think, incorporated in novel ways as we move forward.

Mildred: And Christian, sort of related to that, right, you mentioned in terms of, you know, whether it's regulation or guidance specific to FDA. What would you say about that in terms of, you know, how it goes hand in hand with product liability? ability?

Christian: Absolutely. So, I mean, as we see increased levels of regulation and increased regulatory attention on this topic, I think one aspect that's going to be really critical to keep in mind is that as we are developing our framework here, the regulations that come out are really going to represent, especially at this beginning stage when we're still coming in to better understanding of the technology itself, these regulations are really going to represent the floor rather than the ceiling. And so it's going to be important for companies who are working in this space and thinking about integrating these technologies to think about how can we incorporate and come into compliance with these regulations, but what are the very specific concerns that might be raised above and beyond these regulations? And so, Jamie, you're talking about some of these social media cases where some of the injuries alleged are very specific to subpopulations of of users in these social media platforms. And so how are we, for example, going to address the vulnerabilities in the population that our products are being marketed to while staying in compliance with the regulations as well? And so that sort of interplay is going to be really, really interesting to see to what degree these legal theories are stretched above and beyond what we're used to seeing and how that will impact understanding of the way the regulations are integrated into the business.

Jamie: You raise a great point, Christian, with respect to the regulations being a floor rather than a ceiling. I think there are a lot of companies out there that reasonably think that as long as they're following the regulations and doing what they're supposed to be doing on that front, that their risk in litigation is minimal or maybe even non-existent. But as we know, that's not the case. A medical device manufacturer can do everything right with respect to complying with FDA regulations and still be found liable in a courtroom. You know, plaintiff's lawyers often come up with pretty creative theories to put in front of a jury regarding the number of things the manufacturer, and I have my air quotes going over here, could or should have done but didn't. And these are often things that are not legally required or even practical sometimes. And ultimately, at least with respect to negligence, it's up to the fact finder to decide if the manufacturer acted reasonably. And while this question often involves considerations of whether the manufacturer complied with regulations and guidances and the like, compliance, even complete compliance, is not a bar to liability. And as product liability litigators, we see plaintiffs relying on a lot of the same theories, a lot of the same types of evidence, and a lot of the same arguments. And so having that base of knowledge and being able to share that with manufacturers and say, hey, look, I know we're not there yet. I know this litigation isn't happening today. But here are maybe some things that you can do to help mitigate your potential future risks or defend against these types of cases later on. And, you know, that's one of the reasons why we wanted to start this conversation.

Mildred: Yeah, and I would definitely echo that as well, Jamie, because as Christian mentioned, you know, the guidance that's being put out by FDA, for instance, really that's the floor in many ways and not the ceiling. And sort of looking at the guidance to provide input and insight into, okay, here's what we should be doing with respect to, you know, the design of this algorithm that will then be used for this clinical trial or to deliver this specific type of treatment, right, as you illustrated in the Roots case involving, you know, the allegation of bias in pulse oximeters, for instance, and really looking at mitigating the potential risk that is foreseeable and can be identified. And obviously, not every risk might be identifiable. And that all gets into, you know, the negligence standard in terms of, you know, what is foreseeable and what isn't. But when you're dealing with these very sophisticated, complex technologies, you know, these questions that we're so used to dealing with in sort of your normal product liability case, I think, will get more complex and nuanced. As we start to see these types of cases within the life sciences space, we're already starting to see it within the social media context, as Jamie touched on earlier. And so I would say, you know, because we're getting close to the end here of our podcast in terms of some key takeaways is obviously monitoring the case law, even if it's not in the life sciences space, but for instance, in the social media space and what's going on there as well as other areas. Monitoring what's going on in the regulatory space, because clearly we have a lot going on. And not just FDA, you also have the Federal Trade Commission issuing guidance and speaking to these issues. And if you don't have a prescription, you know, medical device that's governed by FDA, then you most certainly need to look at, you know, what other agencies govern your specific technology, especially if you're partnering with a medical device manufacturer or pharmaceutical company. And of course, legislation, we know there's a lot of activity, both at the federal and the state level, with respect to the regulation of AI. And so I think there too, we have our eye on that as well in terms of, you know, what if any legislation is coming out of that and how it will impact product liability. And Jamie and Christian, I know you guys have other thoughts on, you know, some of the key takeaways.

Jamie: Yeah. So I want to just sort of return to this issue of bias and the importance that manufacturers make sure that they're looking at and taking into account the available knowledge, whether in scientific journals or medical literature, et cetera, related to how factors like race, age, gender. Impact, the medical risks, diagnosis, monitoring, treatment of the condition that a particular device is intended to diagnose or treat, monitoring those things is going to be really important. I also think being really diligent about investigating and documenting the reasons for making certain decisions typically helps. Not always, but usually in litigation, being able to show documentation explaining the basis for decisions that were made can be extremely helpful. So, for example, the FDA put out a guidance document related to a predetermined change control plan, which is something that was developed specifically for medical devices that incorporate artificial intelligence and machine learning. And the plan is intended to set forth the modifications that manufacturers intend to or anticipate will occur over time as the device develops and the algorithm learns and changes post-market. And one of the recommendations in that guidance is that the manufacturer engage with FDA early before they submit the plan to discuss the modifications that will be included. Now, it's not a requirement, but I expect that if a company elects not to do this, that this is something plaintiff's counsel in a products case would say is evidence that the manufacturer was not reasonable, that the manufacturer could and should have talked to FDA, gotten FDA input, but didn't want to do that. Whereas if the manufacturer does do it and there's evidence of discussions with the FDA and even better, FDA's agreement with what the manufacturer ended up putting in its plan, That would be extremely useful to help defend against a product case because you're essentially showing the jury that, hey, this manufacturer talked with FDA, ran the plan by FDA, FDA agreed, the company did what even FDA thought was right, while that wouldn't be a bargain liability, right? Right. It's not it's not going to it's not going to completely immunize a manufacturer, but it is good evidence to support that the company acted reasonably at the time and under the circumstances.

Christian: Yeah. And I would just add to, you know, Jamie, you touched on so many of the important aspects here. I think the only thing I would add at this point is, you know, the importance as well, making sure that you understand the technology that you're integrating. And this goes so well in hand, Jamie, with much of what you just said about understanding who's making the decisions and why. Investing the energy upfront into ensuring that you're comfortable with the technology and how it works will allow you then moving down the line to just be much more efficient in the way that you respond, whether that's to regulatory modifications down the line, whether that's to legal risk. It will just put you in much of a stronger position if you are able to really explain and understand what that technology is doing.

Mildred: And I think that's the key, Christian, as you said, you know, being able to explain how it was tested, that it was robust, right? Yes, of course, it met the guidance, and if there's a regulation, even better. But that all measures, you know, within reasonable balance were taken to ensure that, you know, this technology being used is safe, is effective, and you try to identify all of the potential risks that could be known based on the anticipated way the technology is working. So with that, of course, we can do a whole podcast just on this topic alone with respect to mitigating the risk. And I think that will be a topic that we focus on as part of one of our subsequent podcasts, but I think unless Jamie or Christian, you have any other thoughts that brings us to an end here of our podcast.

Jamie: I think that pretty much covers it. Of course, there's a lot more detail we could get into with respect to the various theories of liability and what we're seeing and the developments in the case law and steps companies can be taking now, but maybe we can save that for another podcast.

Christian: And I completely agree. I think there's going to be so much to dig into over the next few months and years to come. So we're looking forward to it. Thank you, everybody, for listening to this episode of Tech Law Talks. And thank you for joining Mildred, Jamie, and I as we explore the dynamics between AI technologies and the product liability legal landscape. Stay connected by listening to this podcast moving forward. We're looking forward to putting out new episodes talking about AI and other emerging technologies. And we look forward to speaking with you soon.

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's Emerging Technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated. 

Reed Smith and its lawyers have used machine-assisted case preparation tools for many years (and it launched the Gravity Stack subsidiary) to apply legal technology that cuts costs, saves labor and extracts serious questions faster for senior lawyers to review. Partners David Cohen, Anthony Diana and Therese Craparo discuss how generative AI is creating powerful new options for legal teams using machine-assisted legal processes in case preparation and e-discovery. They discuss how the field of e-discovery, with the help of emerging AI systems, is becoming more widely accepted as a cost and quality improvement.

----more----

Transcript:

Intro: Hello, and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day. 

David: Hello, everyone, and welcome to Tech Law Talks and our new series on AI. Over the the coming months, we'll explore the key challenges and opportunities within the rapidly evolving AI landscape. Today, we're going to focus on AI in eDiscovery. My name is David Cohen, and I'm pleased to be joined today by my colleagues, Anthony, Diana, and Therese Craparo. I head up Reed Smith's Records & eDiscovery practice group, big practice group, 70 plus lawyers strong, and we're very excited to be moving into AI territory. And we've been using some AI tools and we're testing new ones. Therese, I'm going to turn it over to you to introduce yourself. 

Therese: Sure. Thanks, Dave. Hi, my name is Therese Craparo. I am a partner in our Emerging Technologies Group here at Reed Smith. My practice focuses on eDiscovery, digital innovation, and data risk management. And And like all of us, seeing a significant uptick in the interest in using AI across industries and particularly in the legal industry. Anthony? 

Anthony: Hello, this is Anthony Diana. I am a partner in the New York office, also part of the Emerging Technologies Group. And similarly, my practice focuses on digital transformation projects for large clients, particularly financial institutions. and also been dealing with e-discovery issues for more than 20 years, basically, as long as e-discovery has existed. I think all of us have on this call. So looking forward to talking about AI. 

David: Thanks, Anthony. And my first question is, the field of e-discovery was one of the first to make practical use of AI in the form of predictive coding and document analytics. Predictive coding has now been around for more than two decades. So, Teresa and Anthony, how's that been working out? 

Therese: You know, I think it's a dual answer, right? It's been working out incredibly well, and yet it's not used as much as it should be. I think that at this stage, the use of predictive coding and analytics in e-discovery is pretty standard, right? Right. As Dave, as you said, two decades ago, it was very controversial and there was a lot of debate and dispute about the appropriate use and the right controls and the like going on in the industry and a lot of discovery fights around that. But I think at this stage, we've really gotten to a point where this technology is, you know, well understood, used incredibly effectively to appropriately manage and streamline e-discovery and to improve on discovery processes and the like. I think it's far less controversial in terms of its use. And frankly, the e-discovery industry has done a really great job at promoting it and finding ways to use this advanced technology in litigation. I think that one of the challenges is that still is that while the lawyers who are using it are using it incredibly effectively, it's still not enough people that have adopted it. And I think there are still lawyers out there that haven't been using predictive coding or document analytics in ways that they could be using it to improve their own processes. I don't know, Anthony, what are your thoughts on that? 

Anthony: Yeah, I mean, I think to reiterate this, I mean, the predictive coding that everyone's used to is it's machine learning, right? So it's AI, but it's machine learning. And I think it was particularly helpful just in terms of workflow and what we're trying to accomplish in eDiscovery when we're trying to produce relevant information. Information, machine learning made a lot of sense. And I think I was a big proponent of it. I think a lot of people are because it gave a lot of control. The big issue was it allowed, I would call, senior attorneys to have more control over what is relevant. So the whole idea is you would train the model with looking at relevant documents, and then you would have senior attorneys basically get involved and say, okay, what are the edge cases? It was the basic stuff was easy. You had the edge cases, you could have senior attorneys look at it, make that call, and then basically you would use the technology to use what I would say, whatever you're thinking in your brain, the senior attorney, that is now going to be used to help determine relevance. And you're not relying as much on the contract attorneys and the workflow. So it made a whole host of sense, frankly, from a risk perspective. I think one of the issues that we saw early on is everyone was saying it was going to save lots of money. Didn't really save a lot of money, right? Partly because the volumes went up too much, partly because, you know, the process, but from a risk perspective, I thought it was really good because I think you were getting better quality, which I think was one of the things that's most important, right? And I think this is going to be important as we start talking about AI generally is, and in terms of processes, it was a quality play, right? It was, this is better. It's a better process. It's better managing the risks than just having manual review. So that was the key to it, I think. As we talked about, there was lots of controversy about it. The controversy often stemmed from, I'll call it the validation. We had lots of attorneys saying, I want to see the validation set. They wanted to see how the model was trained. You have to give us all the documents and train. And I think generally that fell by the wayside. That really didn't really happen. One of the keys though, and I think this is also true for all AI, is the validation testing, which Teresa touched upon, that became critical. I think people realized that one of the things you had to do as you're training the model and you started seeing things, you would always do some sampling and do validation testing to see if the model was working correctly. And that validation testing was the defensibility that courts, I think, latched on on. And I think when we start talking about Gen AI, that's going to be one of the issues. People are comfortable with machine learning, understand the risks, understand, you know, one of the other big risks that we all saw as part of it was the data set would change, right? You have 10 custodians, you train the model, then you got another 10 custodians. Sometimes it didn't matter. Sometimes it really made a big difference and you had to retrain the model. So I think we're all comfortable with that. I think as Therese said, it's still not as prevalent as you would have imagined, given how effective it is, but it's partly because it's a lot of work, right? And often it's a lot of work by, I'll say, senior attorneys instead of developing it, when it's still a lot easier to say, let's just use search terms, negotiate it, and then throw a bunch of contract attorneys on it, and then do what you see. It works, but I think that's still one of the impediments of it actually being used as much as we thought. 

Therese: And I think to pick up on what Anthony is saying, what I think is really important is we do have 20 years of experience using AI technology in the e-discovery industry. So much has been learned about how you use those models, the appropriate controls, how you get quality validation and the like. And I think that there's so much to use from that in the increasing use of AI in e-discovery, in the legal field in general, even across organizations. There's a lot of value to be had there of leveraging the lessons learned and applying them to the use of the emerging types of AI that we're seeing that I think we need to keep in mind and the legal field needs to keep in mind that we know how to use this and we know how to understand it. We know how to make it defensible. And I think as we move forward, those lessons are going to serve us really well in facilitating, you know, more advanced use of AI. So in thinking about how the changes may happen going forward, right, as we're looking forward, how do we think that generative AI based on large language models are going to change e-discovery in the future? 

Anthony: I think there, in terms of how generative AI is going to work, I have my doubts, frankly, about how effective it's going to be. We all know that these large language models are basically based on billions, if not trillions of data points or whatever, but it's generic. It's all public information. That's how the model is based. One of the things that I want to see as people start using generative AI and seeing how it would work, is how is that going to play when we're talking about very, it's confidential information, like almost all of our clients that are dealing with e-discovery, all this stuff's confidential. It's not stuff that's public. So I understand the concept if you have a large language model that is billions and billions of data points or whatever is going to be exact, but it's a probability calculation, right? It's basically guessing what the next answer is going going to be, the next word is going to be based on this general population, not necessarily on some very esoteric area that you may be focused on for a particular case, right? So I think it remains to be seen of whether it's going to work. I think the other area where I have concerns, which I want to see, is the validation point. Like, how do we show it's defensible? If you're going in and telling a court, oh, I use Gen AI and ran the tool, here's the relevant stuff based on prompts, what does that mean? How are we going to validate that? I think that's going to be one of the keys is how do we come up with a validation methodology that will be defensible that people will be comfortable with? Again, I think intuitively machine learning was I'm training the model on what a person, a human being deemed is responsive. So that. Frankly, it's easier to argue to a court. It's easier to explain to a regulator. When you say, I came up with prompts based on the allegations of the complaint or whatever, it's a little bit more esoteric, and I think it's a little bit harder for someone to get their heads around. How do you know you're getting relevant information? So, I think there's some challenges there. I don't know how that's going to play out. I don't know, Dave, because I know you're testing a lot of these tools, what you're seeing in terms of how we think this is actually going to work in terms of using generative AI in these large language models and moving away from the machine learning. 

David: Yeah, I agree with you on the to be determined part, but I think I come in a little bit more optimistic and part of it might be, you know, actually starting to use some of these tools. I think that predictive coding has really paved the way for these AI tools because what held up predictive coding to some extent was people weren't sure that courts were going to accept it. Until the first opinions came out, Judge Peck's decision and the Silvermore and subsequent case decisions, there was concern about that. But once that precedent came out, and it's important to emphasize that the precedent wasn't just approving predictive coding, it was approving technology-assisted review. And this generative AI is really just another form of technology-assisted review. And what it basically said is you have to show that it's valid. You have to do this validation testing. But the same validation testing that we've been doing to support predictive coding will work on the large language model generative AI-assisted coding. It's essentially you do the review and then you take a sample and you say, well, was this review done well? Did we hit a high accuracy level? The early testing we're doing is showing that we are hitting even better accuracy levels than with predictive coding alone. And I should say that it's even improved in the six months or so that we've been testing. The companies that are building the software are continuing to improve it. So I am optimistic in that sense. But many of these products are still in development. The pricing is still either high or to be announced in some cases. And it's not clear yet that it will be cost effective beyond current models of using human review and predictive coding and search terms. And they're not all mutually exclusive. I mean, I can see ultimately getting to a hybrid model where we still may start with search terms to cut down on volume and then may use some predictive coding and some human review and some generative AI. Ultimately, I think we'll get to the point where the price point comes down and it will make review better and cheaper. Right. But I also didn't want to mention, I see a couple other areas of application in eDiscovery as well. The generative AI is really good at summarizing single large documents or even groups of documents. It's also extremely helpful in more quickly identifying key documents. You can ask questions about a whole big document population and get answers. So I'm really excited to see this evolution. And I don't know when we're going to get there and what the price effectiveness point is going to be. But I would say that in the next year or two, we're going to start seeing it creep in and use more and more effectively, more and more cost effectively as we go forward. 

Anthony: Yeah, that's fascinating. Yeah, I can see that even in terms of document review. If a human was looking at it, if AI is summarizing the document, you can make your relevance determination based on the summary. Again, we can all talk about whether it's appropriate or not, but that would probably help quite a bit. And I do think that's fascinating. I know another thing I hear is the privilege log stuff. And again, I think using AI, generative AI to draft privilege logs in concept sounds great because obviously it's a big costs factor and the like. But I think we've talked about this, Dave and Therese, like we already have, like there's already tools available, meaning you can negotiate metadata logs and some of these other things that cut the cost down. So I think it remains to be seen. Again, I think this is going to be like another arrow in your quiver, a tool to use, and you just have to figure out when you want to use it. 

Therese: Yeah. And I think one of the things I think in not limiting ourselves to only thinking about, right, document review, where there's a lot of possibility with generative AI, right, witness kits, putting together witness outlines for depositions and the like, right? Not that we would ever just rely on that, but there's a huge opportunity there, I think, as a starting point, right? Just like if you're using it appropriately. And of course, today's point, the price point is reasonable, you can do initial research. There's a lot of things that I think that it can do in the discovery realm, even outside of just document review, that I think we should keep our minds open to because it's a way of giving us a quicker, getting to the base more quickly and more efficiently and frankly, more cost-effectively. And then you can take a look at that and the person and can augment that or build upon it to make sure it's accurate and it's appropriate for that particular litigation or that particular witness and the like. But I do think that Dave really hit the nail on the head. I don't think this is going to be, we're only going to be moving to generative AI and we're going to abandon other types of AI. There's reasons why there's different types of AI is because they do different things. And I think what we are most likely to see is a hybrid. Right. Right. Some tools being used for something, some tools being used for others. And I think eventually, as Dave already highlighted, the combination of the use of different types of AI in the e-discovery process and within the same tool to get to a better place. I think that's where we're most likely heading. And as Dave said, that's where a lot of the vendors are actually focusing is on adding into their workflow this additional AI to improve the process. 

David: Yeah. And it's interesting that some of the early versions are not really replacing the human review. They are predicting where the human review is going to come out. So when the reviewer looks at the document, it already tells you what the software says. Is it relevant or not relevant? And it does go one step beyond. It's hard because it not only tells you the prediction of whether it's relevant or not, but it also gives you a reason. So it can accelerate the review and that can create great cost savings. But it's not just document review. Already, there's e-discovery tools out there that allow you to ask questions, query databases, but also build chronologies. And again, with that benefit, then referencing you to certain documents and in some cases having hyperlinks. So it'll tell you facts or it'll tell you answers to a question and it'll link back to the documents that support those answers. So I think there's great potential as this continues to grow and improve. 

Anthony: Yeah. And I would say also, again, let's think about the whole EDRM model, right? Preservation. I mean, we'll see what enterprises do, but on the enterprise side, using AI bots and stuff like that for whether it's preservation, collection and stuff, it'll be very interesting to see if these tools can be used there to sort of automate some of the standard workflows before we get to the review and the like, but even on the enterprise side. The other thing that I think it will be interesting, and I think this is one of the areas where we still have not seen broad adoption, is on the privilege side. We know and we've done some analysis for clients where privilege or looking for highly sensitive documents and the like is still something that most lawyers aren't comfortable using. Using AI, don't know why I've done it and it worked effectively, but that is still an area where lawyers have been hesitant. And it'll be interesting to see if gender of AI and the tools there can help with privilege, right? Whether it's the privilege logs, whether it's identifying privilege documents. I think to your point, Dave, having the ability to say it's privileged and here's the reasons would be really helpful in doing privilege review. So it'll be interesting to see how AI works in that sphere as well, because it is an area where we haven't seen wide adoption of using predictive coding or TAR in terms of identifying privilege. And that's still a major cost for a lot of clients. All right, so then I guess where this all leads to is, and this is more future-oriented. Do we think we're at this stage now that we have generative AI that there's a paradigm shift, right? Do we think there's going to be a point where even, you know, we didn't see that paradigm shift bluntly with predictive coding, right? Predictive coding came out, everyone said, oh my God, discovery is going to change forever. We don't need contract attorneys anymore. You know, associates aren't going to have anything to do because you're just going to train the model, it goes out. And that's clearly hasn't happened. Now people are making similar predictions with the use of generative AI. We're now not going to need to do docker view, whatever. And I think there is concern, and this is concern just generally in the industry, is this an area, since we're already using AI, where AI can take over basically the discovery function, where we're not necessarily using lots of lawyers and we're relying almost exclusively on AI, whether it's a combination of machine learning or if it's just generative AI. And they're doing lots of work without any input or very little input from lawyers. So I'll start with Dave there. What are your thoughts in terms of where do we see in the next three to five years? Are we going to see some tipping point? 

David: Yeah, it's interesting. Historically, there's no question that predictive coding did allow lawyers to get through big document populations faster and for predictions that it was going to replace all human review. And it really hasn't. But part of that has been the proliferation of electronic data. There's just more data than ever before, more sources of data. It's not just email now. It's Teams and texts and Slack and all these different collaboration tools. So that increase in volume is partially made up for the increase in efficiency, and we haven't seen any loss of attorneys. I do think that over the longer run that there is more potential for the Gen AI to replace replace attorneys who do e-discovery work and, frankly, to replace lawyers and other professionals and all other kinds of workers eventually. I mean, it's just going to get better and better. A lot of money is being invested in. I'm going to go out on a limb and say that I think that we may be looking at a whole paradigm shift in how disputes are resolved in the future. Right now, there's so much duplication of effort. If you're in litigation against an opposing party, You have your documents set that your people are analyzing at some expense. The other side has their documents set that their people are analyzing at some expense. You're all looking for those key documents, the needles in the haystack. There's a lot of duplicative efforts going on. Picture a world where you could just take all of the potentially relevant documents. Throw them into the pot of generative AI, and then have the generative AI predetermine what's possibly privileged and lawyers can confirm those decisions. But then let everyone, both sides of court, query that pot of documents to ask, what are the key questions? What are the key factual issues in the case? Please tell us the answers and the documents that go to those answers and cut through a lot of the document review and document production that's going on now that frankly uses up most of the cost of litigation. I think we're going to be able to resolve disputes more efficiently, less expensively, and a lot faster. And I don't know whether that's five years into the future or 10 years into the future, but I'll be very surprised if our dispute resolution procedure isn't greatly affected by these new capabilities. Pretty soon, I think, when I say pretty soon, I don't know if it's five years or 10 years, but I think judges are going to have their AI assistance helping them resolve cases and maybe even drafting first drafts of court opinions as well. And I don't think it's all that far off into the future that we're going to start to see them. 

Therese: I think I'm a little bit more skeptical than Dave on some of this, which is probably not surprising to either Dave or to to Anthony on this one. Look, I think, I don't see AI as a general rule replacing lawyers. I think it will change what lawyers do. And it may replace some lawyers who don't keep pace with technology. Look, it's very simple. It's going to make us better, faster, more efficient, right? So that's a good thing. It's a good thing for our clients. It's a good thing for us. But the idea, I think, to me, that AI will replace the judgment and the decision-making or the results of AI is going to replace lawyers and I think is maybe way out there in the future when the robots take over the world. But I do think it may mean less lawyers or lawyers do different things. Lawyers that are well-versed in technology and can use that are going to be more effective and are going to be faster. I think that. You're going to see situations where it's expected to be used, right? If AI can draft an opinion or a brief in the first instance and save hours and hours of time, that's a great thing. And that's going to be expected. I don't see that being ever being the thing that gets sent out the door because you're going to still need lawyers who are looking at it and making sure that it is right and updating it and making sure that it's unique to the case and all the judgments that go into those things are appropriate. I do find it difficult to imagine a world having, you know, been a litigator for so many years where everyone's like, sure, throw all the documents in the same pod and we'll all query it together. Maybe we'll get to that point someday. I find it really difficult to imagine that'll happen. There's too much concern about the data and control over the data and sensitivity and privilege and all of those things. You know, we've seen pockets of making data available through secure channels so that you're not transferring them and the like, where it's the same pool of data that would otherwise be produced, so that maybe you're saving costs there. But I don't, again, I think it'll be a paradigm shift eventually in that, paradigm shift that's been a long time coming, though, I think, right? We started using technology to improve this process years ago. It's getting better. I think we will get to a point where everyone routinely more heavily relies on AI for discovery and that that is not the predictive coding or the tar for the people who know how to use it, but it is the standard that everybody uses. I do think, like I said, it will make us better and more efficient. I don't see it really replacing, like I said, entirely lawyers or that will be in a world where all the data just goes in and gets spit out and you need one lawyer to look at it and it's fine. But again, I do think it will change the way we practice law. And in that sense, I do think it'll be a paradigm shift. 

Anthony: The final thought is, I think I tend to be, I'm sort of in the middle, but I would say generally we know lawyers have big egos, and they will never allow, they will never think that a computer, AI tool or whatever, is smarter than they are in terms of determining privilege or relevance, right? I mean, I think that's part of it is, there's, you know, you have two lawyers in a room, they're going to argue about whether something is relevant. You have two lawyers in a room, they're going to argue about something privileged. So it's not objective, right? There's subjectivity. And I think that's going to be one of the chances. And I think also, we've seen it already. Everyone thought. Every lawyer who's a litigator would have to be really well-versed in e-discovery and all the issues that we deal with. That has not happened. And I don't see that changing. So unless I'm less concerned about being a paradigm shift than all of us going out for those reasons. 

David: Well, I think everyone needs to tune back in on July 11th, 2029 when we come back to get stuff to begin and see who we're going. 

Anthony: Yes, absolutely. All right. Thanks, everybody. 

David: Thank you. 

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's emerging technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.

Singapore is developing ethics and governance guidelines to shape the development and use of responsible AI, and the island nation’s approach could become a blueprint for other countries. Reed Smith partner Bryan Tan and Raju Chellam, editor-in-chief of the AI Ethics & Governance Body of Knowledge, examine concerns and costs of AI, including impacts on owners of intellectual property and on workers who face job displacement. Time will tell whether this ASEAN nation will strike an adequate balance in regulating each emerging issue.

----more----

Transcript:

Intro: Hello and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with everyday. 

Bryan: Welcome to Tech Law Talks and our new series on artificial intelligence. Over the coming months, we'll explore all the key challenges and opportunities within the rapidly evolving AI landscape. Today, we'll focus on AI and building the ecosystem here in Singapore. My name is Bryan Tan, and I'm a data and emerging technology partner at Reed Smith Singapore. Together, we have with us today, Mr. Raju Chellam, the Editor-in-Chief of the AI E&G BOK, And that stands for the AI Ethics and Governance Body of Knowledge, initiative by the SCS, Singapore Computer Society, and IMDA, the Infocomm Media Development Authority of Singapore. Hi, Raju. Today, we are here to talk about the AI ecosystem in Singapore, of which you've been a big part of. But before we start, I wanted to talk a little bit about you. Can you share what you were doing before artificial intelligence appeared on the scene and how that has changed after we now see artificial intelligence being talked about frequently? 

Raju: Thanks, Bryan. It's a pleasure and an honor to be on your podcast. Before AI, I was at Dell, where I was head of cloud and big data solutions for Southeast Asia and South Asia. I was also chairman of what we then called COIR, which is the Cloud Outage Incidence Response. This is a standards working group under IMDA, and I was vice president in the cloud chapter at SCS. In 2018, the Straits Times Press published my book called Organ Gold on the illegal sale of human organs on the dark web. I was then researching the sale of contraband on the dark web. So all of that came together and helped me when I took over the role of AI in the new era. 

Bryan: So all of that comes from dark place and that has led you to discovering the prevalence of AI and then to this body of knowledge. So the question here is, so tell us a little bit about this body of knowledge that you've been working on. Why does it matter? Is it a game changer? 

Raju: Let me give you some background. The Ethics & Governance Body of Knowledge is a joint effort by the Singapore Computer Society and IMBA, the first of its kind in the Asia-Pacific, if not the world, to pull together a comprehensive collection of material on developing and deploying AI ethically. It is anchored on the AI Governance Framework 2nd Edition that IDA launched in 2020. The first edition of the BOK was launched in October 2020 before GenAI emerged on the scene. The second edition focused on GenAI was launched by Minister Josephine Thieu in September 2023. And the third edition, the most comprehensive, will be launched on August 22, which is next month. The most crucial thing about this is that it's a compendium of all the use cases, regulations, guidelines, frameworks related to the responsible use of AI, both from a developing concept as well as a deploying concept. So it's something that all Singaporeans, if not people outside, would find great value in accessing. 

Bryan: Okay. And so I see how that kind of relates to your point about the dark web, because it is really about a technology that's there that can be used for a great deal of many things. But without the ethics and the governance on top of that, then you run into that very same kind of use case or problem that you were researching on previously. And, you know, So as you then go around and you speak with a lot of people about artificial intelligence, what do you really think is the missing piece or the missing pieces in AI? What are we not doing today? 

Raju: In my view, there are two missing pieces in AI, especially generative AI. One is the need for strong ethics and governance guidelines and guardrails to monitor, if not regulate, the development and deployment of AI to ensure it is fair, transparent, accountable, auditable. Two, is the awareness that AI, especially GenAI, can be used just as effectively by bad actors to do harm, to commit crimes, to spread fake news and even cause major social unrest. So, these two missing pieces which are not mutually exclusive can be used for good as well as bad. It's the same with the beginning of the airplanes, for instance. Airplanes can be used to ferry people and cargo around the world. They can also be used to drop bombs. So we need strong guardrails in place. And the EU AI Act is just a starting point that has shown the world that AI, especially GenAI, needs to be regulated so that companies don't misuse information that customers and businesses entrust to it. 

Bryan: Okay. Let's just move on a little bit. about cybersecurity. Some of your background is also getting involved with cybersecurity, advising, consulting on cybersecurity. In terms of generative AI, do you see any negative impact, any kind of pitfalls that we should be looking out for from a cybersecurity point of view? 

Raju: That's a very pertinent question, given that the Cyber Security Agency of Singapore has just released data that estimates that 13% of phishing scams might be AI-generated. There are also two darker versions of ChatGPT, for example. One is called Fraud GPT, F-R-A-U-D, and the other is called Worm GPT, W-O-R-M. Both are available on the dark web. They can also be used for RAAS, which is ransomware as a service that bad actors can hire to carry out specific attacks. Being aware of the negative possibilities of GenAI is the first step for companies and individuals to be on guard and keep their PII or personally identifiable information safe. So as a person involved in cybersecurity, I think the access that bad actors have to the tool that's so powerful, so all-consuming, so prevalent, can be a weapon. 

Bryan: And so it's an area that we all need to kind of watch out for. You can't simply ignore the fact that alongside the tremendous power that comes with the use of GenAI, the cybersecurity aspects should not be ignored. And that's something we should pay attention to. But other than just moving away from cybersecurity, other than cybersecurity, any other issues in AI that also worry you? 

Raju: The two key concerns about AI, according to me, other than cybersecurity, are number one, the potential of AI to lead to a loss of jobs for humans. And the second concern is its impact on the environment. So let me delve a little deeper. The World Economic Forum has estimated that AI adoption could impact 85 million jobs by 2030. Goldman Sachs has said in a report that AI could replace about 300 million full-time jobs. McKinsey reports that 14% of employees might need to change their careers due to AI by 2030. This could cause massive unrest in countries with large populations like India, China, Indonesia, Pakistan, Brazil, even the US. The second is sustainability. According to the University of Massachusetts at Amherst study, the training process for a single AI model can emit 284 tons of carbon dioxide. That's equal to greenhouse gas emissions of roughly 62.6 petrol-powered vehicles being driven for a year in the US. These are two great impacts. People, governments, companies, regulators have yet to grapple with because these could become major issues by the time we turn this decade. 

Bryan: So certainly some challenges coming up. I remember that for many years you were also an editor with the Business Times here in Singapore. And so this question is about media and media content, specifically, I think, digital media content. And, you know, with that background in mind, now looking closely at generative AI, do you see generative AI affecting the area of digital media and content generation? Do you see any interesting use cases in which gen AI has been applied here? 

Raju: Yes, I think digital media and content, including the entire field of advertising, public relations, marketing, will be or is being currently impacted to a large extent by Gen AI, both in its use as well as in its potential. To the extent that many digital media content companies are actively looking at GenAI as a possible route to replace human labor. In fact, if you look at the Hollywood Actors Union, they all went on strike because producers were turning to GenAI to even come up with movie scripts. So, it is a major concern because unlike previous technologies which impacted the lowest ranks of the value chain, such as secretarial jobs, for instance. GenAI has the potential to impact the higher or highest value chain, for instance, knowledge workers. So they could be threatened because all of their accumulated knowledge can be used by GenAI to churn out material as good as, if not better than, what humans could do in certain circumstances. Not in all circumstances, but with digital media content, most of the time, the GenAI model is not augmenting its human potential, it's also churning out material that can be used without human oversight. 

Bryan: So certainly a challenge and interesting use case in the field of digital media content. Last question, and again, back to the body of knowledge and talked a little bit about the Singapore government's involvement in this area. In Singapore, we do have a tendency for a lot of things to be government-led. In this particular area where we are really talking about frontier technology like artificial intelligence. Do you think this is the right way to go about it to let the government take the lead? And if so, what more can be done or should be done? 

Raju: That's a good question. The good part is that Singapore is probably one of the very few countries, if not the only one where the government tries to be ahead of the curve in tech adoption and in investing in cutting-edge technologies such as AI, quantum computing, biotech, etc. While this is generally good in the sense that a clear direction is set for industry to focus on, is there a risk that companies may focus too narrowly on what the government wants instead of what the market wants? I don't know. More research needs to be done in this area. But look at the numbers. Spending on AI-centric systems is set to surpass 300 billion US dollars worldwide by 2026, as per IDC estimates, up from about $154 billion in 2023. So Singapore's focus on AI and GenAI was the right horse to bet on. And it's clear that AI is not a fad, not a hype, not an evolution, but a revolution in tech. So at least we got that part right here. Whether we will get the other parts or the components right, I think only time will tell.

Bryan: Okay, and final question, looking at it from ecosystem point of view, various moving parts, various parts working together. For you personally, if you had a crystal ball and a wishing wand and you could wish for anything in the future that would help this ecosystem or you think will aid this ecosystem, what would that be? 

Raju: I think there is need for stronger guardrails and some kind of regulation to ensure that people's privacy is protected. The reason is, GenAI can infringe upon the copyrights and IP rights of other companies and individuals. This can lead to legal, reputational, and or financial risks for the companies using pre-trained models. GenAI models can perpetuate or even amplify biases learned from the training data, resulting in biased, explicit, unfair or discriminatory outcomes, which could cause social unrest if not monitored or audited or accounted for accurately. And the only authority or authorities that can do this are government regulators. So I think government has to take a more proactive role in ensuring that basic human rights and basic human data is protected at all times. 

Bryan: With this, I thank you. Certainly a lot more to be done in building up the ecosystem to encourage and evolve the role of AI in today's world. But I want to thank you, Raju Chellam, for joining us. And I want to invite you who are listening to continue to listen to our series of Tech Law Talks, especially this one on artificial intelligence. And thank you for hearing us. 

Raju: Thank you, Bryan. It's been a pleasure. 

Bryan: Likewise. Thanks so much, Raju. I really enjoyed doing this. 

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's emerging technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.

Regulatory and investigations partner Kendra Perkins Norwood invites former U.S. GSA Associate Administrator Krystal Brumfield to discuss how the federal government is gaining an understanding of AI’s uses in procurement. They also explain how the General Services Administration and other federal agencies are using AI to streamline and safeguard the contract award process.

----more----

Transcript:

Intro: Hello and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day. 

Kendra: Hello, welcome to Tech Law Talks, a new Reed Smith podcast series on artificial intelligence or AI, as it's commonly called. I am Kendra Norwood, a partner in Reed Smith's Global Regulatory and Investigations Practice Group. I'm based in Washington, D.C., and my specific practice area is government contracts. Through the Tech Law Talk series, we will be exploring the key challenges and opportunities that are presented within the rapidly evolving AI landscape, and today we will focus on AI in government contracts. For today's episode, we are very fortunate to have a special guest joining us, Krystal Brumfield, who for the past three and a half years has served as the Associate Administrator for the Office of Government-Wide Policy at the General Services Administration, or GSA. Now, for those who may not be familiar with GSA, it's an independent agency of the U.S. Government that provides centralized procurement and shared services for the federal government, managing a nationwide real estate portfolio, overseeing over $100 billion in federal government contracts for goods and services purchased by the government, and for delivering technology services to millions of both government and public users across dozens of federal agencies. So when we decided to do a podcast on AI and government contracting, Krystal immediately came to mind as the perfect podcast guest to discuss that topic, and I am so glad to have her here today. So welcome, Krystal.

Krystal: Thank you, Kendra. Excited to be here with you and Reed Smith.

Kendra: Wonderful. So if you could start off with a brief introduction of yourself and your role and responsibilities at GSA, that would be great to set the stage for our discussion today.

Krystal: Sure. So I served as the Associate Administrator for Government-Wide Policy in January 2020 through May 2023. In that role, I was designated as the Regulatory Policy Officer, the chief acquisition officer. I also served as the chair of the agency Cyber Supply Chain Risk Management Executive Committee. And as the associate administrator of OGP, it's what we call government-wide policy for short, I oversaw the drafting and promulgation of the federal acquisition regulation, also known as the FOG. IT policy, as well as the Federal Acquisition Institute, which is responsible for training and educating the 20,000 plus federal acquisition professionals across the federal government.

Kendra: Oh, wow. That is such an impressive background and just some really big responsibilities. And again, I think you're the perfect guest to have here today to talk about this topic. So again, thank you so much for being here. So I guess I'd like to start by framing our discussion a bit. I tend to think about AI in government contracting in two ways. So first, I think about how the government uses AI to support the federal acquisition process, and that's from contract formation to contract administration, and ultimately through contract closeout, and using AI in that way to work smarter, faster, and more efficiently. And the second way I think about government contracts AI is the approach used by the government to purchase AI-driven tools through government contracts in order to support agency missions. We are seeing more and more federal government contracting opportunities that are either in whole or in part being released for the acquisition of AI of some type. And so I'm hoping we can address both of those uses of AI during our time today.

Krystal: So, Kendra, you absolutely have described it correctly. It's kind of a two-pronged approach from the way that we see it or saw it in my role as the associate administrator at GSA. We saw it so much so used in our everyday work, really across agency, that once the executive order President Biden issued on safe, secure, and trustworthy development and use of artificial intelligence, the agency decided to hire a chief AI officer because it was that important of a function across the agency. As you mentioned, it's really, it is sort of a new phenomenon when it comes to government contracting. And so one of the other things that we saw when when we are doing procurements is that there was a gap in understanding and knowledge between where we needed to be in the use of AI and the acquisition workforce. And so while I was there, we stood up the GSA Acquisition Policy Federal Advisory Committee, which what we call GAPFAC for short, which is a federal advisory committee that consists of federal, state, local, and government officials, representatives from trade associations. Professors from universities, as well as business leaders from all across the country to really help us, answer problems related to federal contracting. And one of those problems is AI, understanding AI, what it means to the business of federal contracting. And that's one of the key areas that we saw was important. It was a growing trend and it's ever evolving. And so our GAPFAC very soon will be focusing on AI in procurement.

Kendra: Oh, that's wonderful. I had not heard about GAPFAC, but I love the way it seems as if it's using a very collaborative and cross-sector approach with individuals involved sort of at all levels of government, across the government, and even in the private sector. So that sounds really exciting. I look forward to hearing more about what comes out of that. So I guess let's just dive right in as we talk about how AI is used to support the federal acquisition process. Now, during your time at GSA, were there any specific milestones or key developments related to the use of AI other than what you've already mentioned, which is phenomenal, in order to improve contracting procedures?

Krystal: Absolutely. So on the government side, GSA started using artificial intelligence for the pre-award vendor assessment about two years ago. So just to kind of break it down to the listeners of how it works in a very simplified way. You would gather data, relevant data, about potential vendors from various sources. And so this data could include historical performance data, financial records, customer reviews, compliance records, all from available public databases or information that's provided from the vendors themselves. This data is then integrated into a centralized system or platform where our AI algorithms can assess and analyze it. This could involve cleaning the data and standardizing the data to make sure that it's consistent and it's accurate. Then we would move to the scoring and ranking phase where the algorithms would generate scores and rankings for each of the vendors based off of the extracted features and historical data. The scores will reflect the likelihood of the vendors meeting our specific criteria or performing well in the context of the contract that's being awarded. Then we would move to the decision phase where. Final output that we would see would be an AI analysis providing whoever the decision maker is with valuable insights and recommendations, helping the procurement officer or the project manager make a more informed decision on who to invite to further evaluate or negotiate the contract with. And so we've seen this process work many times, the benefits being that we gain more efficiency in the government. It helps us to reduce some of the automated or arduous tasks that we have. We've also experienced there being more accuracy. And so we're reducing more human error throughout the process. As you can imagine, there are lots and lots of contracts, could be a lot of modifications to those contracts. And so to help scale that volume down, AI has been helping in that way. And then it also creates an opportunity for us to rely on data-driven decisions rather than subjective judgments. And so this helps a lot in a lot of different ways. But one of the things that we've been careful about is making sure that it's essential that we ensure that we train the AI models on diversity and representative data sets so that we can manage the bias that data sometimes has, as well as ensuring that we have fair evaluations throughout the process.

Kendra: So that's pretty impressive, Krystal. I'm sort of awestruck right now because it sounds like to me that for all practical purposes, AI is being used to handle pretty much the entire evaluation process up to the point of making those recommendations to the selecting official. And I guess I'm wondering, is there any involvement or is there still involvement by a source selection board or some other human element in this process before the recommendation goes to the source selection official?

Krystal: Absolutely. That human component can't be replaced. Because we know that AI is ever-evolving, it's a new phenomenon that we're still trying to understand. And it is instances where there has created error and biases. Then the human eyes and perspective and analysis remains a key component to keep as a part of the process.

Kendra: Well, that sounds like a win-win. I mean, you know, there could be errors with AI, but of course, as you mentioned, there are often human errors. I mean, it sounds like AI could be used to reduce those. And I would imagine perhaps reduce the amount of protests that we see, which isn't good for my business, but I think overall good for the government if we can sort of eliminate or at least minimize or mitigate that human error factor. So that's great to know. So just moving along, you know, as I understand it, there's two basic categories of AI. So there's traditional AI, which is great for, you know, addressing sort of well-defined problems, performing repetitive tasks, and dealing with very structured data. And then there's generative AI, which is used to work with more unstructured data and sort of designed to learn new content. You've already explained how some of that is already happening in federal government agencies, GSA in particular, sort of using that traditional AI to automate certain manual tasks across the entire government contracts lifecycle cycle. I know Department of Health and Human Services uses it to consolidate certain contract vehicles. I previously worked at NASA. You're at GSA. I know both of those agencies are using traditional AI to deploy robotic processes, basically bots, chatbots that are software-based robots to execute standard rules-based business procedures and interface with the users, the system users. And so, it's sort of that kind of traditional AI, I think that does have the potential to reduce. Workload backlogs, which, as I understand, it can be substantial depending on the agency, in addition to helping agencies conduct procurements more quickly, which is what I think I heard you say as you described the system that's already in place now. I know the Air Force was at one time contemplating using AI to help acquisition professionals better understand these very complex procurement policies, rules, regulations, again, towards towards speeding up that process, which can often be very lengthy and is often something that discourages some companies from wanting to do business with the government just because of the involved process. So it seems as if AI could certainly be used to help that. Now, these are just a few examples. And, you know, there's some who believe that, you know, this kind of traditional AI could be used to completely automate, you know, sort of those early contracting procedures, You know, deciding what type of contract should be used, what contract type should be used, how it should be structured, should it be set aside for small business. You know, they've said that GSA multiple award schedule contracts, the blanket purchase agreements, IDIQ task order contracts, GWACs, the government-wide acquisition contracts, all of those are sort of in many ways specific to GSA And I'm just wondering if you have any thoughts on how specific to those contract types AI could apply.

Krystal: Sure. So, I mean, Kendra, these are all great examples of how traditional AI can be used and has been used to improve and automate procurement processes within the federal government organizations. There are countless ways and things that we can point to that have have been beneficial to using AI. But I also want to make sure that it's important that we keep in mind that by using AI to automate these routine tasks, for example, the market research and the pre-solicitation period that I mentioned earlier, or the contract modifications, invoicing, or the award-free determinations, there can be some pretty substantial financial impacts for the government or even the contractor or both if errors occur around pricing or payments. And so there's all, but there's concerns that we have sometimes when we rely on AI alone, when it's reading regulations, right? It could lead to regulations being misinterpreted in some cases or even possibly misapplied. And so if that happens, it could have some opposite effects of slowing down a procurement process. In ways that we don't anticipate there could be. And so whether it's through increased bid protests or the need to redo procurements when AI generated errors or discovered. Or even if there are a number of unintended consequences with using AI that we know about or don't know about, this by no means is a perfect solution. So we have to weigh the pros and cons when it comes to it, because although there are a lot of benefits to it, there are certain things that we don't know. And in some cases, there are certain errors or inefficiencies that it may cause.

Kendra: Those are all great points. So, you know, as many benefits as AI bring to the table, you know, There are, again, associated risks, and it sounds like the government is taking that into account and factoring that into its use of AI by not eliminating the human element. So I guess turning to generative AI, I was thinking how this could be used to, in some instances, allocate or manage risk. I know that, for instance, it's already being used to collect data points to determine if If a contractor or a prospective contractor is presently responsible, that's a term of art in government contracts, as I'm sure you know, and you have to be presently responsible to be eligible to receive a government, a federal government contract award. So that's, you know, sort of that use of collecting disparate data and bringing it together to make determinations on responsibility. Also, maybe schedule and risk assessments, you know, determining whether, you know, a project is likely to be completed on time, but at the same time, as you mentioned, on the flip side, could have some consequences in terms of setting up some unrealistic expectations to the extent AI isn't factoring into some of the very real considerations that go into whether a project is completed on schedule. Now, again, these are just hypotheticals, well, except for the one about the responsibility determinations. But again, in terms of the generative AI, can you speak briefly on that, the use of that in the government?

Krystal: So I think what you kind of laid out there are all great examples. And we know these and even, in fact, other applications for AI are just around the corner if they aren't already in use. But GSA has really been at the cutting edge and the lead, and we've long recognized the power of generative AI to increase efficiencies, to lower our operating costs, and even to prevent and detect some criminal activity against the federal government. My office, our top priority was to make government efficient. More modern, streamlined, and accessible. That was our North Star, that we drove all of our policies behind that and our drive to make government operations more efficient and effective was to make sure that we're modernizing the way that we're doing things. And the regulations would reflect that, that we were streamlining them to make sure that processes disease were more efficient and they were accessible to all. And so with that in mind, we think that there is power and great benefit from generative AI. A couple of examples that we saw was with creating documents for contracting officials. We also saw we are utilizing pattern recognition and trend analysis of financial data to identify fraud activity in federal financial systems. Using generative AI to create sample data sets that could be used to test software or even customize commercial software for government use. Also, the cybersecurity threat detection could use AI by using it to model trained historical cyber data like network traffic or user interactions so that they could anticipate and respond to the cyber attacks against our federal IT systems, which of course we often know contain very sensitive information related to government contracting, whether that's financial data or confidential and proprietary data that belongs to companies doing business with the federal government agency, but it resides in government systems. So all of these examples really just show the benefit of generative AI. I think the applications are limitless in terms of how AI can improve our operations or the government's operations and also how they can better deliver efficiencies all across the federal government.

Kendra: Wow. I mean, as you said, the possibilities are limitless. You know, it's just the power of data and the power that AI brings to the table in terms of leveraging that data to make things more efficient and more mission focused, as you mentioned, for federal agencies. So just quickly, I want to touch on how the government is going about purchasing these tools, these AI tools that they're using to use traditional regenerative AI in their day-to-day work. I've seen solicitations come along here lately that are for the purchase of AI tools, whether that's the entirety of the procurement or AI is still somewhat embedded as an expectation or an option for a contractor to propose when they are selling or attempting to sell to the government. Now, I guess the biggest thing that comes to mind for me is that have there been any ethical concerns that factor into how the government is going about procuring these AI technologies? And if so, how are they being addressed?

Krystal: Yeah, well, I believe that there are some fundamental requirements that the government only procures AI technology. That it both use, you know, do so responsibly, but also trustworthily. So first, we have to recognize that AI is one of the most profound technological shifts in this generation. And because the space is so large, and because it has so many complexities, I think contracting officers, they should consider cybersecurity. Supply chain, risk management, data governance, and other standards and guidelines when it comes to procuring AI, just as they would any other IT procurements. I think it's critical that our acquisition workforce also work with and consult with the technical subject matter experts like software engineers and data scientists. You know, the good news is that there are several efforts that are underway already to ensure that the ethical AI standards are in place and they exist. For example, GSA and the Department of Defense's Joint Artificial Intelligence Center have a center of excellence effort in place and have had it for several years now that has a focus on advancing AI technology across DOD. The center of excellence has a guide to AI ethics that it promotes the development of ethical AI by adopting AI applications with a human-centered mindset and approach. The guide also includes a series of questions that should be answers to every phase of an AI development project to ensure that there are ethical designs, developments, and deployments within the AI solution, along with any other extensive testing to mitigate unintended consequences from the application of this AI technology. The Department of Defense also has its own policy document on ethical principles of AI. And of course, the president has issued an executive order. That really addresses and really is the foundation of which all of these different policies rest with regards to being in promoting the use of trustworthy artificial intelligence across the federal government. And so absolutely should be ethical considerations. And I think we are seeing a lot more information come out about considerations to think about or require when it comes to ethics.

Kendra: Wow. So it just seems like there's a lot going on, but it also seems like it's being done in a very intentional and thoughtful way in the government, which is reassuring. And I'm sure my clients, our customers that are doing business with the federal government, whether they are selling AI technologies to the government or implementing AI technologies in order to better sell to the government or better perform on government contracts. And so it just, you know, it's encouraging to see that the government is already in many respects leveraging the power of AI in both purchasing as well as in its administration of government contracts. So I guess the last point I want to touch on speaking to sort of my client base is sort of what should government contractors keep in mind as they consider how to incorporate AI into their business models?

Krystal: Sure. Well, companies who are doing business with the government or even those who are interested in entering into the government procurement market space, they should make sure that their company's vision and goals are aligned with the technological evolution that's occurring around AI. I mean, identifying the AI tools that can lead to future business opportunities is essential, I think, for the private sector. I also think that it's also important to consider what business objectives can be better achieved through the use of AI. And just for businesses in general, they should be mindful that AI is going to continue to grow in the government space. It's not going to go anywhere. And so companies have to keep up with that. And they want to be a part of the government solution. They have to prepare for it. And so I think to jump in the game, They got to get involved. They got to be a part of the game. And so they've got to adopt AI as a part of their practice and as a part of their protocol and as a priority in their business.

Kendra: Well, that's very well said. I couldn't agree more. And I think we are right at the point where I want to thank you so much for your time and your contributions today. I think this has been really enlightening for me and hopefully for the listeners as well. Thank you so much. And then we will continue to look forward to everything that comes along with AI in the federal government space.

Krystal: Thank you, Kendra. Appreciate the conversation today.

Kendra: Likewise. Thank you so much for listening and have a great day.

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's emerging technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.

Can we keep up with AI? Paul Foster, CEO of the Esports Federation, dives into the legal implications of artificial intelligence. Gamers have a unique familiarity with artificial intel. Explore how AI is transforming game design, content creation, brand promotion and much more. Along with entertainment/media lawyer Bryan Tan of Reed Smith’s Singapore office, Foster discusses the unique ways AI is enabling gamers to monetize their skills. 

----more----

Transcript:

Intro: Hello and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day. 

Bryan: Welcome to Tech Law Talks and our new series on artificial intelligence. Over the coming months, we'll explore the key challenges and opportunities with the rapidly evolving AI landscape. Today, we will focus on AI in the interesting world of esports. And we have together with us today, Mr. Paul Foster, who is the CEO of the Global Esports Federation. Good morning, Paul. 

Paul: Good evening, Bryan. It's nice to be with you, coming from California. 

Bryan: And I'm coming to you from Singapore, but we are all connected in one world. Today, we are here to talk about AI and esports. But before we start, I wanted to talk about you and to share what you were doing before AI and how that has changed after AI has now become a big thing. 

Paul: Thanks, Bryan. Yeah. So I come from originally from Sydney, Australia, and I was from my background is really 20 years in the Olympic movement. So I started at the Sydney 2000 Olympics. So I started about three years before the Olympics, worked for the Olympics for about 20 years, so traditional sports, and then moved to esports and opened or founded the Global Esports Federation in 2019. And of course, we all know what happened four months after that, Bryan, with COVID and the pandemic that really closed a lot of traditional sport. And so esports really took off. So it's definitely been a very, very exciting and accelerating journey these last couple of years. 

Bryan: Great. So you've kind of gotten into esports after a background in sports. And it's interesting you mentioned 2019 because we also know that somewhere around 2022, the end of 2022, just as the pandemic was sorting itself out. Artificial intelligence, primarily generative artificial intelligence, then began to capture the imagination of people. And the question here, I guess, is maybe we're talking to converted, but esports is obviously one of the most technologically advanced and clued in online ecosystems, communities. How has artificial intelligence impacted esports and is that a positive or negative thing? 

Paul: You're absolutely right, Bryan. I think one of the things that I like to say is that we're living in what I consider to be one of the most exciting times in the history of humanity. The reason I say that is because of the convergence of all these incredibly powerful technologies at exactly the same time, at the very early dawn of AI. AI and I think it's something that we should reflect on because I think many people talk about AI as if we're already in the middle of the cycle and I think my position is that we're at the very early dawn, maybe even the pre-dawn of AI. I did some postgraduate studies in machine learning and AI so it's a passion project of mine, something I'd love to think about and I was recently at the global summit on AI with the International Telecommunications Union in Geneva and had a chance to sit with the leaders group. There was 30,000 people attending, Bryan, which is a big number of people showing the interest from all over the world. But there was a leaders group convened to look at policymaking. And in a sense, there was this feeling that industry has been rapidly growing and expanding almost at a pace or a cadence that is hard to sort of register in a sense. And then policymakers and particularly governments and others are trying to sort of catch up in a sense and and try to get in front of that. As you know Bryan we're very strong partners with UNESCO, the United Nations Education Science and Cultural Organization, as well as the international telecommunications union so we're also contributing to their thinking and bringing our community into the discussion one of the things that could be interesting for our listeners is that it's true what Bryan said is that our community, which is roughly in the range of 18 to 34-year-olds, are early adopters of technology. And one of the things that might be interesting is that gamers and people who play electronic sports and games have always been exposed to artificial intelligence in some form, even very, very early form. And so it's not a surprise to me that the adoption of of artificial intelligence and the interest in artificial intelligence in some ways has been really accepted by the esports community in the gaming community and as you also said Bryan you know this this demographic 18 to 34 is really the heart of what we call Gen Z or Gen and then now we're seeing Gen Alpha of course the next generation starting high school but what we also call this and some people widely call this generation is actually Gen T. Generation around around technologies and the early acceptance of that. And I can talk a little bit more about some of the applications for esports if that's interesting. 

Bryan: No, I think that's interesting. And I think in particular, I think what will be interesting to hear is some of your own visions about what the future of AI and esports could be like. What does it promise to the esports community? What can they kind of look forward to? How do you see that going? 

Paul: Yeah I think thanks for the question I think that it's um again I think there's the the possibilities are limitless and we're really at the beginning the early time about this and when I speak for example recently to colleagues and friends at companies such as open ai and others that I speak to every day there's a true interest around this particularly around the the creative economy and how we'll create games in the future and there's three things Bryan that I thought I'd mentioned, which is really the use of AI in terms of teams and players' preparation, the fact that we can have quicker and more efficient, I call it, you know, one of the great benefits, one of the things I've learned in my studies is that it needs to be human-focused and human-centric. And we're also, at the Global League Sports Federation, we support the UNESCO's position on the creation of ethical AI. And what that really talks about is human-focused because it's human-centered. And so one of the things I think is really interesting for our community is that they'll have quicker access to statistics, to analytics, to data. So they'll be able to, if you think about esports as a competitive sport or as a competitive event, any preparation that you can have to prepare you to have better results and better preparation will ultimately, should ultimately provide for a better outcome for you as a competitor, right? So that's number one. The second thing is that really what we can do for the creative economy, which is absolutely fascinating, Bryan, in esports, the whole economy around or the whole community around creators and content creators and people that really bring esports and bring it alive. Is that we'll be able to have automatically created clips and reels and analytics. So in real time, things like in pre-AI, we would have had to wait for editing, Bryan. We would have had to wait for editing. it might have taken hours or days and now that can happen in seconds so for and so that's fascinating and then the third thing so team preparation creative economy and then the third thing really talks around the economics of gaming and around sponsorship and value-based identity so that in the future our sponsors and our partners that are so important for the thriving nation and the the sustainability of gaming and esports will also be able to use AI to have greater analytics and greater awareness of their brand values, to actually understand the value of their brand. A very simple example is we can use AI to track how many times a certain brand was visible on a jersey or in an audience. We'll be able to use AI to actually track it in real time. Whereas again, Bryan, in the past, we would have had to look through video files and actually count it manually. And I remember doing that. I mean, another example I'll give you is I remember but not that long ago in my work at the Olympic Games, I literally remember installing what we used to call video walls. So walls of not even a video screen, but 12 video screens or 16, I think they were, 4x4 screens to be able to look at every venue at once. And when I think about that now, that seems like a long time ago, but it wasn't a long time ago. I mean, within the last 10 or 12 years, that was still our reality. And now we can use AI to capture that data, to give us the same results in real time across teams, across creators and across partners. 

Bryan: Okay, thank you for that. I think three very concrete areas that we can look forward to as an esports community. Interestingly, you also mentioned the regulators, the governments trying to keep in touch with the development of AI. Yes. And it sounded as if it was a bit of a struggle for them. Do you think there are any big concerns about the deployment of AI esports that we should be kind of aware of and maybe try to avoid? 

Paul: Yeah, I think it's really this notion about catching up, right? How do you catch up with something that's evolving every day and every hour of every day at a speed that's really difficult to contain? And also two schools of thought, really, which is one school of thought, which I remember, Bryan, I think Sam Altman from OpenAI recently said it. And he said, look, we're so busy and this is running so fast and so powerful, we'll come back and we'll get back to that later. Like we're off, you know, creating these incredibly strong and powerful platforms. We'll have to come back to those matters at a later stage. And it was interesting because when I was last couple of weeks ago in Geneva, you had policymakers, governments, ministers, etc., whose role was to make sure that the frameworks were established around implementing the framework on ethical AI. Were really struggling with this reality of being able to just, I mean, literally physically struggling with this reality of trying to get ahead of the knowledge, not only the knowledge, but also the policy work that needed to be put in place, the frameworks, the regulations, and then rolling that out across industry. At the same time, you have technology firms and particularly firms with specialization in AI, and you've seen the incredible value chain skyrocketing in recent months, really racing ahead. And yet you've got policymakers trying to get their hands around this and trying to even understand it. You've got the same challenges in academia, don't you, Bryan, with academia also trying to create curricula that by the time it's published, we may already be behind the eight ball in terms of where AI has taken us. So I think the thing that I would talk about is the concern I would have is the ethical side of AI because, you know, and keeping it human focused and in the best interest of humanity, meaning that really what the benefits are, the focus of benefits should be around making our lives more efficient, effective and more equitable. And there is a risk of course within AI that it can because of prejudice that is potentially built into the ai itself that it could continue to manifest that across the community and that's something that's difficult to get ahead of unless it's created with that lens at the very beginning. 

Bryan: No i think that's that's absolutely correct it's uh it's a good reminder that this is technology we're dealing with, and technology can be something that's used for the good of humanity, but it can also be abused. And we have to keep in mind that the technology is there only for the benefit of mankind, like you said, and to keep that human centricity always in focus as AI is applied to esports. Okay, so last question, I promise you, Paul, as CEO of Global Esports Federation, what would you wish for the future of esports? And maybe just to make it interesting, on two spectrums, one, a more realistic expectation, and the second one, a moonshot. If your wish can be granted, what would your wish for esports be? 

Paul: What a great question. Thanks, Bryan. I love that opportunity. Well, Well, the thing that's so interesting in esports and gaming is that anything that was a moonshot about two weeks ago is now already a reality. It moves so fast. So when you were mentioning a moonshot, I was thinking about the Olympics. And I'll talk about that in a moment because that would have been considered a moonshot just a few months ago, if not years ago. But what I think the future is, is the globalization of esports as a source of incredibly inclusive, powerful, evocative entertainment, right? So just as you have traditional sport and just as you have, for example, in the United States, you have the proliferation of leagues and professional sports. It's coming into view that you'll have very significant value and be able to really create a very sustainable living as an economic means through esports. Not only as a player winning significant prize money, but also as a content creator, as a game developer, as a marketeer, as an event organizer, as an academic. There's tons of opportunities. opportunities and in fact Bryan I was speaking with some friends of mine who are attorneys actually and it surprises me because traditionally I would talk to attorneys and then through conversation it comes out that they're really passionate about gaming and now maybe they specialize around being with illegal expertise in terms of intellectual property rights or different aspects of it and this also I wanted to share that with you Bryan because I thought that was interesting that even in a traditional professional, such as the practice of law. There's now a lot of interest in this field as well. What does that mean? That means that we get to manifest our lives how we wish them to be manifested. In the past, if I wanted to go into event management, I would have to do a certain angle. Now I can do that with inside esports. If I wanted to be in communications and global media, I might have had to do that in public relations, or in traditional luxury goods, for example, or consumer products. Now I can do it inside esports. So I think the future is extremely bright and relatively limitless in terms of being able to manifest my career, finding something I want to do in my profession, my skills, but be able to do it in something that I love doing. And that's a blessing, I think, Bryan, that very few of us, so you and I, that has happened in our lifetime, that we're able to actually have the life that we want, create professional professional conditions we want, earn a living of that by doing it in that field that we love. The moonshot which you've challenged me on, I was so proud having come from the Olympic movement in my hometown of Sydney and now seeing the reality of the Olympic eSports Games, which was just announced by the IOC a couple of weeks ago and then rapidly evolving. And how interesting is this? At the Paris 2024 Olympic Games, it seems that we'll see an announcement of the Olympic eSports Games itself, agreed by the IOC, confirmed by the IOC. And so one of the things i think is fascinating if you think about Olympic sport traditional sport, it took golf 121 years to come back onto the Olympic program and esports in global esports federation was as you said Bryan, founded in 2019 here we are just four years later not only is it inside the Olympic movement but there's actually a separate IP created called Olympic esports games. If we think about that for a moment, the IOC traditionally had their Olympic Games as their main IP. Yes, Winter Games. Yes, Youth Games. But now we have the Olympic Esports Games as a separate IP. And what's even interesting at the recent press release I read is that it said a whole new division, a whole new structure will be created at the IOC. Rather than trying to fit it into traditional models, a whole new structure will be created. it. So this was a moonshot. And I think that this will be fascinating in terms of how we see that evolve and how you see a traditional sports organization just a couple of years ago, really being a long way away from today. And in those very short years with the Global Esports Federation staging our Commonwealth Esports Championships, the European Esports Championships, the Pan American Esports Championships, and now seeing the evolution at the Olympic Esports Games, What an incredible opportunity that is for athletes, creators and community right around the world. 

Bryan: Thank you for sharing that. And I think that's a great statement to make that what was yesterday's moonshot is today's reality in a fast-paced world that evolves because of technology. Thank you again for sharing with us your thoughts, Paul. I think it's been greatly exciting. We look forward to a great future in esports. And once again, thank you for joining us in this series. 

Paul: Thank you, Bryan. Thanks very much, everyone. 

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith's emerging technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.

AI offers new tools to help competition enforcers detect market-distorting behavior that was impossible to see until now. Paris Managing Partner Natasha Tardif explains how AI tools are beginning to help prevent anticompetitive behaviors, such as collusion among competitors, abuse of dominance and merger control.

----more----

Transcript:

Intro: Hello and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day. 

Natasha: Welcome to our new series on AI. Over the coming months, we'll explore the key challenges and opportunities within the rapidly evolving AI landscape. Today, our focus is going to be on AI and antitrust. AI is at the core of antitrust authorities' efforts and strategic thinking currently. It brings a number of very interesting challenges and great opportunities, really. In what ways? Well, let's have a look at how AI is envisaged from the perspective of each type of competition concept. I.e. anti-competitive agreements, abuse of dominant position, and merger control. Well, first of all, in relation to anti-competitive agreements. Several types of anti-competitive practices, such as collusion amongst competitors to align on market behavior or on prices, have been assessed by competition authorities. And when you look at those in relation to algorithms and AI, it's a very interesting area to focus on because a number of questions have been raised and some of them have answers, some others still don't have answers. The French competition authorities and the German Bundeskanzleramt issued a report sharing their thoughts in this regard in 2019. The French competition authority went as far as creating a specific department focusing on digital economy questions. At least, three different behaviors have been envisaged from an algorithm perspective. First of all, algorithms being used as a supporting tool of an anti-competitive agreement between market players. So the market players would use that technology to coordinate their behavior. This one is pretty easy to apprehend from a competition perspective because it is clearly a way of implementing an anti-competitive and illegal agreement. Another way of looking at algorithms and AI in the antitrust sector and specifically in relation to anti-competitive agreements is when one and the same algorithm is being sold to several market players by the same supplier, creating therefore involuntary parallel behaviors or enhanced transparency on the market. We all know how much the competition authorities hate enhanced transparency on the market, right? And a third way of looking at it would be several competing algorithms talking, quote-unquote, to each other and creating involuntary common decision-making on the market. Well, the latter two categories are more difficult to assess from a competition perspective because, obviously, we lack one essential element of an anti-competitive agreement, which is, well, the agreement. We lack the voluntary element of the qualification of an anti-competitive agreement. In a way, this could be said to be the perfect crime, really, as collusion is made without a formal agreement having been made between the competitors. Now, let's look at the way AI impacts competition law from an abuse of dominance perspective. In March 2024, the French Competition Authority issued its full decision against Google in the Publishers' Related Rights case, whereby it fined again Google for €250 million for failing to comply with some of its commitments that had been made binding by its decision of 21 June 2022. The FCA considered that Bard, the artificial intelligence service launched by Google in July 2023, raises several issues. One, it says that Google should have informed editors and press agencies of the use of their contents by its service Bard in application of the obligation of transparency, which it had committed to in the previous French Competition Authority decision. The FCA also considers that Google breached another commitment by linking the use of press agencies and publishers' content by its artificial intelligence service to the display of protected content and services such as search, discover, and use. Now, what is this telling us about how the competition authorities look at abuse of dominance from an AI perspective? Well, interestingly, what it's telling us is something it's been telling us for a while when it comes to abuse of dominance, and particularly in the digital world. These behaviors have even been so much at the core of the competition authorities', concerns that they've become part of the new digital markets app. And this DMA now imposes obligations regarding the use of data collected by gatekeepers with their different services, as well as interoperability obligations. So in the future, we probably won't have these Google decisions in application of abuse of dominance rules, but most probably in application of DMA rules, because really now this is the tool that's been given to competition authorities to regulate the digital market and particularly AI tools that are used in relation to the implementation of the various services offered by what we now call gatekeepers, big platforms on the internet. Now, thirdly, the last concept of competition law that I wanted to touch upon today is merger control. What impact does AI have on merger control? And how is merger control used by competition authorities to regulate, review, and make sure that the AI world and the digital world function properly from a competition perspective? Well, in this regard, the generative AI sector is attracting increasing interest from investors and from competition authorities, obviously, as evidenced by the discussions around the investments made by Microsoft in OpenAI and by Amazon and Google in Anthropic, which is a startup rival to OpenAI. So the European Commission considered that there was no ground investigating the $13 billion investment of Microsoft in OpenAI because it did not fall under the classic conception of merger control. But equally, the Commission is willing to ensure that it does not become a way for gatekeepers to bypass merger controls. forms. So interestingly, there is a concern that the new way of investing in these tools would not be considered as a merger under the strict definition of what a merger is in the merger control conception of things. But somehow, once a company has been investing so much money in another one, it is difficult to think that it won't have any form of control over its behavior in the future. Therefore, the authorities are thinking of different ways of apprehending those kind of investments. The French Competition Authority, for instance, announced that it will examine these types of investments in its advisory role, and if necessary, it will make recommendations to better address the potential harmful effects of those operations. A number of acquisitions of minority stakes in the digital sector are also under close scrutiny by several competition authorities. So, again, we're thinking of situations which would not give control in the sense of merger control rules currently, but that still will be considered as having an effect on the behavior and the structure of those companies on the market in the future. Interestingly, the DMA, the Digital Markets Act, also has a part to play in the regulation of AI-related transactions on the market. For instance, merger control of acquisitions by gatekeepers of tech companies is reinforced. There is a mandatory information of these operations, no matter the value or size of the acquired company. And we know that normally for an information to be given to the competition authorities, it would be the notification system that is only required where certain thresholds are met. So we are seeing increasing attempts by competition authorities to look at the digital sector, particularly AI, from different kinds of lenses, being innovative in the way they approach it because the companies themselves and the market are being innovative about this. And competition authorities want to make sure that they remain consistent with their consumptions and concepts of competition law while not missing out on what's really happening on the market. So what has the future really made us now? Well, the European Union is issuing its Artificial Intelligence Act, which is the first ever comprehensive risk-based legislative framework on AI worldwide. That will be applicable to the development, deployment and use of AI. It aims to address the risks to health, safety and fundamental rights posed by AI systems while promoting innovation and the outtake of trustworthy AI systems, including generative AI. The general idea on the market and from a regulatory perspective is that if you're looking at competition law or more generally as a society, when you're scrutinizing AI, even though there may be abusive behavior through AI, the reality of it is AI is a wonderful source of innovation, competition, excellence on the market, added value for consumers. So authorities and legislators should try to find the best way to encourage, develop, nurture it for the benefits of each and every one of us, for the benefits of the market and for the benefits of everybody's rights really. Therefore, any piece of legislation or case law or regulation that will be implemented in the AI sector must be really focusing on the positive impacts of what AI brings to the market. Thank you very much. 

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith’s Emerging Technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.

Gregor Pryor of Reed Smith’s Entertainment & Media Group in London describes why it’s important for law firms to train their lawyers in how to use AI. Although AI-powered tools do not exceed living lawyers in all aspects of legal practice, their powers of calculation bring immense yields in efficiency and can be a powerful accelerator for law firms delivering services.

----more----

Transcript:

Intro: Hello, and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode of this podcast, we will discuss cutting-edge issues on technology, data, and the law. We will provide practical observations on a wide variety of technology and data topics to give you quick and actionable tips to address the issues you are dealing with every day. 

Gregor: Hi, everybody, and welcome to our new series on artificial intelligence. Over the coming months, we'll explore some of the key challenges and opportunities within the rapidly evolving AI landscape. And today we're going to focus on artificial intelligence within law firms and hopefully give a bit more context about how Reed Smith is deploying artificial intelligence in the delivery of its services to clients. The first thing I would say is that certainly in my day job as an entertainment lawyer, AI has been a very controversial subject, mostly as it pertains to training. And as we'll discover as I explain more about how Reed Smith is using AI, training and the ability for law firms to use data that it obtains, is highly contingent on clients agreeing to that training or being comfortable with the manner in which law firms are deploying the technology to improve and make their services more efficient. So the first thing I want to talk about is how we are using AI and what our future plans are. So there's a whole lot of buzz and hype, I think, about how AI is impacting the way that law firms are operating. There's been a number of surveys. phase, most of them say that AI will transform the business of law and having been in the business of law for 25 years. I've heard that for the whole 25 years, but now feels like the time when it actually is happening and the decisions that firms are making to adopt AI are having an impact on how they perform. Most of the global 200 firms have policies about how they use generative AI. There's a high level of governance and risk management concerning client data, as I mentioned. But not all of those firms have a policy concerning how they use AI. We've been working and have worked on ours for a number of years and continue to iterate as the technology improves and client perspectives on the use of AI change. We think that there's likely a gap between preparedness and managing risk and the implementation of AI and we've been being very careful as we prepare our infrastructure to integrate and use AI carefully. Obviously the bigger the law firm, the more they are able to leverage AI and invest, but not that many firms are working with clients on AI projects. We've just finished a trial of about seven different providers. We've used them on a beta basis through limited rollout. We're not putting all our eggs in one basket. We're trying to figure out which AI has proper utility, which machines generate real-time efficiencies and help us in the delivery of our service. I think it's fair to say that some of them are nowhere near as impressive as we'd hoped, but we are still continuing to invest. One of the things that we've been very careful about is organizing our data and making sure it's hygienic. That means not using client data for AI without permission and also making sure that we have organized ourselves so that our data doesn't get unnecessarily or incorrectly commingled. One of the other topics that comes up is how AI can improve efficiency and productivity. I think there's some really obvious ways, such as summarizing documents, helping translate things, creating text or drafting. It gets a bit more complex. I think decision tree software has been around for a long time, but that's evolved so that you have a much more of a chatbot style interaction with AI. Of course, how do we address ethical, security, confidentiality concerns? These are all going to be developed in conjunction with clients. We don't think that we have the right or privilege of dictating to our clients how we use technology. But we do want to be a first mover where we can be because that will give us an advantage over other providers, other firms. We think that training our lawyers how to use AI is almost as important as the technology itself. It's no use having these incredible tools, but not being able to have lawyers that are well-versed in using them. And indeed, that is a trend that we're seeing within our clients as well. So we've been delivering prompt training, use case training for lawyers, because unless lawyers themselves change the way they work and adopt the use of the machines, then the machines will be pretty useless. One of the things I often get asked is whether I perceive AI as a competitive advantage for law firms. I'm not sure that it's necessarily only AI. And I do think there are quite tight limits on the use of AI. One bit of feedback we had from a particular client was that they preferred their lawyers to understand how the law works. And just reading case summaries generated by AI was no substitute for human learning. And to a great extent, I agree with that. Although I do think that there are ways in which lawyers learn and the change in the way that we practice over the past 20 years has been huge. I remember, I'm making myself seem very old, but I remember going to photocopying rooms and delivering faxes and carrying around big bundles of documents. Things that seem very arcane to us now in the early 20s, but actually when I qualified 20 years ago they were real things that we had to do. We've seen some clients use AI in really interesting and clever ways. We're fortunate that many of the clients we represent, huge technology companies who already have very sophisticated artificial intelligence applications and utilities. So that gives us an insight into how they expect us to adapt and align with them. One of the challenges we face is that exact issue, given that many of our clients have their own thoughts, processes, ethics, rollout, timetables for AI, and we have to align our delivery with what they're doing. And then finally, I guess one of the things that I should talk about is the limitations of AI. What can it do and what can it not do? Where would we struggle to use it? I still think when it comes to advocacy and negotiation. Particularly live and in real time, there's still a very strong place for lawyers with talent, with a keen intellect, who are quick on their feet, and who can see through a client's objectives very capably. I don't see a computer doing that anytime soon. We can look at case predictors or outcome predictive models, certainly to try and streamline the transactional process or find a way to reduce litigation costs. But having lawyers that can think however powerful the AI is to human brains is typically much more powerful. And so long as we are able to leverage, educate ourselves, use the technology to our advantage, and leverage it for a better outcome for our clients, then I think AI can be an incredibly powerful accelerator for us in the delivery of our services at retail. Whether there are any particularly outlandish or wild predictions related to the use of AI in the next decade or so. I could see a couple of things. Firstly, I could see is for low-level legal services. The use of “robolawyers” or entirely automated services to conclude transactions or effectuate things that you want to do as a consumer or an SME, almost certainly that's going to happen we're already seeing it in the venture capital space we're already seeing it with some of the automated services you see coming out of silicon valley why would you pay hundreds of pounds of dollars or thousands for a lawyer when you can just have a machine that would give you an outcome that's if you know provided you're prepared to accept a little bit of risk I could see robolawyers being a standalone service that's one thing I could see happening. The other thing that I could see happening within the next decade is the abolition or at least firms operating with complete abandonment of an hourly rate. It's something that Reed Smith has been examining closely. We've got client value teams. We've got a bunch of alternative billing models. We give clients options and choices about how they might want to pay for what we're delivering. But for law firms, there's a real opportunity there. If we've invested in and can leverage technology to deliver something much more quickly or to a better standard than a competitor, it doesn't necessarily follow that we would charge less. It may be that we can deliver it for a lower cost and our margin would increase, but that doesn't necessarily mean we're going to price it. There's a race to the bottom on pricing. So I'm quite bullish on some of the opportunities that are afforded by AI if a firm can gain a competitive advantage. If everyone's using the same technology, then inevitably the prices will go down because clients will see, why would I pay more if you're all using the same thing? So the prize, if you like, will be for law firms who can figure out ways to create products or services more efficiently and to a higher standard, and then still be able to charge well for them and increase their margin. So law firms typically have a high margin, they're high margin businesses. We don't want to be greedy. We want to deliver great value to clients. But equally, if there are opportunities to make money through the use of AI, we'll pursue them. And I think one of the ways to do that is to really critically challenge and question the use of the hourly rate, which has a bunch of built-in inefficiency. 

Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and Shannon Ryan. For more information about Reed Smith’s Emerging Technologies practice, please email [email protected]. You can find our podcasts on Spotify, Apple Podcasts, Google Podcasts, reedsmith.com, and our social media accounts. 

Disclaimer: This podcast is provided for educational purposes. It does not constitute legal advice and is not intended to establish an attorney-client relationship, nor is it intended to suggest or establish standards of care applicable to particular lawyers in any given situation. Prior results do not guarantee a similar outcome. Any views, opinions, or comments made by any external guest speaker are not to be attributed to Reed Smith LLP or its individual lawyers. 

All rights reserved.

Transcript is auto-generated.