Tag: Machine Learning

Pitfalls in AI?

Artificial Intelligence (AI) is the fastest-growing eDiscovery solution in the Legal Industry. Just like in Henry Ford’s day, it’s the keen cutting edge shaving away costs by reducing time spent from evidence to production. Use AI and don’t land in the pitfall.

“Competition is the keen cutting edge of business, always shaving away at costs”…Henry Ford

Is there a pitfall if you use AI? Computer Forensic Experts Lee Neubecker interviews Chief Innovation Office with DISCO, Cat Casey both agree the largest pitfall in AI is NOT embracing AI! Artificial Intelligence (AI) is the fastest-growing eDiscovery solution in the Legal Industry. Just like in Henry Ford’s day, it’s the keen cutting edge shaving away costs by reducing time spent from evidence to production.

Cat explains DISCO was born out of the firm’s frustration with conventional eDiscovery tools that were slow and difficult for lawyers to use. Instead of being forced to adapt our work methods to technology, we wanted to invent technology that works the way lawyers work. DISCO was the result, and today we are the fastest-growing eDiscovery solution in North America. Both experts agree implementing AI will help companies gain a competitive edge. Watch this video to hear examples of how AI helps sharpen that edge!

Final Part of our 3-Part Series in Artificial Intelligence: Pitfalls in AI

The Video Transcript Pitfalls in AI Follows.

Lee Neubecker (LN): Hi and welcome back again Cat. Thanks for being on the show again.

Cat Casey (CC): My pleasure.

LN: Cat Casey from CS Disco. She’s a Chief Product Innovation Officer. Did I say that right?

CC: Chief Innovation Officer.

LN: Okay.

CC: Products too, though. It’s fine.

LN: They call her chief.

CC: They should.

LN: So we’re going to talk now, in this last part of our series on artificial intelligence, about some of the challenges of organizations that don’t adapt and don’t get on board. So, what do you see the potential risks and pitfalls for law firms that don’t begin to embrace so sort the form of a technology-assisted review or artificial intelligence to help speed up the review process?

CC: Well, at a very basic level, clients are getting smarter. We’ve got CLOC https://cloc.org/, we’ve got clients talking to each other more, and they’ve raised their expectations of how their firms are going to be competitive. And it used to be if you were big law firm A you would always have this corporate client for every anti-trust case they would always go to you. But now I was getting dozens of RFPs where they’re asking me what technology are you using? How are you driving innovation? How are you driving efficiency? Because there is a higher expectation of competition between outside counsel. That, maybe, wasn’t there a few years ago. And so, the client expectation is driving this appetite to investigate eDiscovery and Artificial Innovation (AI) based innovation in a way that wasn’t here a few years ago.

LN: Has there been any industry research that has attempted to benchmark the cost of a case using an AI platform to speed up review versus not, to your knowledge?

CC: You know. I can speak from Disco, and we see about a 60% reduction in time to evidence to production. And that translated to dollars. And so, I mean, 60% savings on the 80% of a case that is reviewed is substantial. The thing that I think is most important is cost-savings big, but getting evidence quicker.

LN: Yeah. Time is of the essence.

CC: That is the thing that is paramount because of a lot of these companies… I worked at a company that had very big budgets, but no amount of money, no amount of people, was going to be enough to get these insights I needed before the meet and confer. Or before I had a critical filing with a government investigator. And so, getting evidence quicker so I can start building my case, was the differentiator.

LN: Yeah, certainly if you’re working for a company facing a DOJ inquiry.

CC: Yep.

LN: Knowing the good, the bad, the ugly.

CC: Yep.

LN: As soon as possible can help you make better decisions for your clients. Which might involve, you know, settlement. settling. Yeah, yeah. There have been many recent settlements, recently, from big companies that didn’t want to get tied down at least.

CC: Well I’ve had cases where… One of my favorite ones I used tons of different AI and analytic tools. I had a big bank that had been fined billions of dollars and another big bank was, they had hired on people in that same group, and they were wondering if they would be subject to the same investigation. So, I did some social network analysis. Who was talking to who, with what frequency? I parsed Bloomberg’s chat. I parsed audio logs. And I used everything to keep triangulating down until I was able to identify the bad actors, saying the bad things, and the map of the structured data to show they didn’t do the bad things. And my company wasn’t on the front page of the Wall Street Journal. My company wasn’t fined. So it ends up being very compelling, even early in investigations.

LN: Yeah. Certainly responding quickly is important now. Have you seen any success stories as it relates to companies embroiled with data breach incidences, that have used your platform to help get ahead of what was going on?

CC: 100%. I mean PII, so personally identifiable information, is something that you’re going to have to notify if there is a breach. So if someone, say your Equifax, not that I’m naming them, but say you’re a big company with a lot of personally identifiable or health information. You need to identify it quickly, notify these people in their specific timelines. Tools, like Disco’s, help you use algorithms to find that quickly and act upon it. Otherwise, if you’re looking at 100 million records, there’s no amount of humans that could go through that, in a timely manner, where you’re going to comply with time obligations. And so, it’s majorly impactful.

LN: That certainly is. Well, are there any other things you want to say on the show before we wrap up?

CC: You know, adapt. The reality is no one wants to be the buggy whip maker in a Tesla world. The time to start investigating and vetting and ensuring that the tech you’re looking at isn’t hype is now. Because in a year, or three years, or four years, you might be behind the curve. So, find your resident dork, ask questions, dig into the tech. Now is the time.

LN: And it’s probably worthwhile, you know, without being biased towards Lit Funder, why not take a case try out Disco, try out another offering to see what really works. I mean you had the benefit of…

CC: Yeah.

LN: You were on the other side working for the law firm, shopping for vendors.

CC: I did a 55 vendor RFP. I’ve seen everyone. I’ve looked under every hood. I mean there’s a reason I went to Disco. But there are other tools good out there. I think you want a toolbox with lots of different tools. If you’re a hammer, everything looks like a nail. Let’s be honest, litigation is always bespoke, so you want lots of tools that can help you address it.

LN: Great. Well, thanks again for being on the show.

CC: Yeah, my pleasure.

LN: This was great.

Watch the Entire Series on Artificial Intelligence (AI)

Part 1 of our 3-Part Series on AI

Re-inventing Legal Technology: Artificial Intelligence (AI)
Part 1 in our 3-Part Series on AI

Part 2 of our 3-Part Series on AI

AI Trends in the Legal Industry
Part 2 on our 3-Part Series

Other Related Articles

AI Smarter Solutions: eDiscovery
Optimize eDiscovery With AI

DISCO’s website

https://www.csdisco.com/about-us

The Association for the Advancement of Artificial Intelligence

http://www.aaai.org/

Optimize eDiscovery With AI

You’re looking for the smoking gun and have tens of thousands of documents to review. Experts Lee Neubecker and ZyLAB’s eDiscovery Director, Jeffrey Wolff say Optimize with AI and make your review easier!

Optimize eDiscovery with AI! Lee Neubecker sets out on a quest to find out what’s happening with Artificial Intelligence as it relates to the eDiscovery review process. Lee visits eDiscovery Director, Jeffrey Wolff from ZyLAB and together they examine how new AI algorithms are coded for priority review and can rank documents for relevance, saving countless hours and dollars for the client. Utilizing new AI will optimize your current eDiscovery process.

Part 2 of 3 Part Series on Smarter Solutions eDiscovery

Part 2 of our 3 Part Series on Smarter Solutions in eDiscovery

Optimize eDiscovery with AI Video Transcript Follows

Lee Neubecker (LN): Hi, I have Jeff Wolff back on the show again from ZyLAB. Jeff, thanks for coming back.

Jeff Wolff (JW): Thank you.

LN: And today we’re going to talk a little bit more about trends in Artificial Intelligence as it relates to eDiscovery and the review process that comes along with that. Jeff, what do you see happening right now with Artificial Intelligence as it relates to the eDiscovery review process?

JW: So what we’ve noticed over time is that, traditionally, Artificial Intelligence was always deemed to be only valid in cases where you had hundreds of thousands or millions of documents. And one of the changes that have happened over the last few years is that the Artificial Intelligence models have gotten so much better than you can now use them for much smaller data sets, and so we evangelize the use of Artificial Intelligence in smaller data sets, even, a thousand documents, you’re going to get a better review, more efficient, and more correct, faster, with AI than you would with a team of reviewers.

LN: So if you have a project and you’re using your platform, let’s say there are a million pages of documents that need to be reviewed. You put a review team on starting that process, and they start categorizing and coding, as they get through the first ten thousand documents, what is your software doing to help make this process more efficient and effective for them?

JW: Sure, so if you’re using traditional, what we call supervised machine learning, that used to be referred to as predictive coding, what our software allows you to do is train a small training batch, so a small sample of the documents, and code them for responsiveness, whether they’re responsive or not responsive. And we’ve made it very easy for users to do that. So, you can create issues, and for each issue, you get two tabs, responsive or not responsive, and you just train, you look through a bunch of training documents and you tag the documents appropriately, and the machine classifier learns, very quickly, what is responsive, what is not responsive. So, maybe after two or at most three training batches, the classifier is now bringing you back almost exclusively responsive documents. It’s already smart enough to do that. And so you only need a few training rounds to get the classifier well over the 80%, typical 80% precision and recall threshold that most attorneys feel is what the human is capable of, but the machine will do 90, 95% precision and recall, so you can be assured, not only are you getting a more efficient and more correct review, but you’re also doing it in a whole lot less time with a whole lot fewer people.

LN: And so, are your algorithms looking for synonyms, and similar phrasing that has equivalent word matches?

JW: It’s a bit of secret sauce. But, yeah, we use a support vector machine-based set of algorithms, kind of the most modern version of machine learning. And it is effective, it understands what our topics that were identified in the document, and what other topics are like them. So that’s how it’s doing an identification. But you’re effectively training in or on that.

LN: So the people using your platform, are they having to necessarily review all of the documents, or are you basically, based on the trained review process, you’re taking that universe of a million, and as they get through it, it’s starting to cluster.

JW: Correct.

LN: There’s a set that, this probably isn’t useful, and you don’t have to look at it, but you can look through it just to see.

JW: Sure.

LN: They have confidence that it’s not excluding relevant stuff, right?

JW: Yeah. What we find from an AI standpoint is that the two primary use cases that attorneys have when they use AI are priority review, so that means hey, I’m going to start teaching the data about, the classifier about my data set, and I’m going to show what responsive documents look like, and then I want it to rank all the remaining documents for me for relevance. And so I’m going to then put eyes on those top-ranking documents. That’s effectively looking for the smoking gun, right? That’s one. But they also use it a lot for QC and this is where I see I’m trying to put a lot more attorneys into utilizing AI, is you’ve already done your tagging, and you had eyes on all of your documents, now go back and use the AI and compare it against what your human reviewers did, and see if you’ve missed things. Because inevitably, your reviewers are not going to be all at the same level. Some people are going to miss-tag documents, and the AI has a really good chance of picking up those mistakes and showing them to you.

LN: So have there been any published studies that document the effectiveness of AI with the review process?

JW: There’s been a bunch of them. I know Law Geeks did one that was pretty interesting. What I’ve read recently is that only about, nationally, about 4% of all cases use Artificial Intelligence officially. But then again, there’s no requirement, in the meet and confer that you identify that you are using Artificial Intelligence in a discovery case. So a lot of attorneys can be used, and just not reporting it. Which is fine, because back when the review was manual, and you went through paper and bankers boxes, you didn’t have to document the process for that review. So why should you have to document the fact that you using a machine to do some of the identification of documents and responsiveness today?

LN: So are there potential problems as a result of using AI for failing to produce relevant documents?

JW: No, I think the case law already demonstrates that AI is an accepted form of using, of identifying reviewed documents, and again, even if you’re just using it for QC purposes, you’re still better off. You’re still less likely to miss things than if you hadn’t used it at all.

LN: Great, well, it’s been great. Thanks a bunch for being on the show.

JW: My pleasure, my pleasure.

View Part 1 of our 3 Part Series on Smarter Solutions in eDiscovery

AI Smarter Solutions: eDiscovery
Part 1 of our 3 Part Series about Smarter Solutions and eDiscovery

Other Articles about Artificial Intelligence (AI)

Re-inventing Legal Technology: Artificial Intelligence (AI)
AI Trends in the Legal Industry

More related articles

Medical Device Security Challenges

To Learn More about ZyLAB’s Ability to Optimize eDiscovery With AI

https://www.zylab.com/

AI Trends in the Legal Industry

AI trends in the Legal Industry is revolutionizing data, and whittling down the amount of paperwork involved in legal practice. Lee Neubecker and DISCO’s Cat Casey discuss trends in the legal industry.

Paper death! Legal professionals get buried in a mountain of paperwork. Artificial Intelligence (AI) replaces that mountain of paper with cloud-based apps and whittles down costs. What’s new in Artificial Intelligence (AI) as it relates to the legal industry? Check out this video as Forensic Expert Lee Neubecker and DISCO’s Information Officer Catherine “Cat” Casey talk through AI trends in the legal industry.

View Part 2 of our 3 Part Series on Artificial Intelligence (AI) in the Legal Industry

Artificial Intelligence (AI) in the Legal Industry

The video transcript AI Trends in the Legal Industry follows:

Lee Neubecker: Hi, I’m back here again with Cat Casey from CS Disco. Thanks for coming back again.

Cat Casey: My total pleasure.

LN: We’re going to continue our conversation in this multipart series. This time, we’re talking about artificial intelligence and the trends impacting the legal industry and the whole eDiscovery industry as well.

CC: Absolutely, so in my role at Disco, I’m chief innovation officer, and one of the things I’m tasked with doing, both now and in my prior roles, is going out and figuring out what’s going on in the market, and what we’re seeing is AI written everywhere. Sometimes it’s true AI, sometimes it’s not, but what we are seeing is people want to find evidence faster. People want to eliminate those low-hanging tasks that aren’t the practice of law. And so, we’re seeing a lot of tools that are driving efficiency both in practice management and litigation management and in finding evidence.

LN: So where do you see we’ve gone in the last few years with AI in terms of advancements and providing products for the review process?

CC: When we first, I think, announced AI about 2006, seven, eight, nine, I was working as a channel partner with the company that patented the word predictive coding. That was the first AI model in eDiscovery and people liked it. They didn’t really want to use it. They were nervous. What I’ve seen is not only has the process improved instead of TAR 1.0, where you have a sample, you make decisions, and then, the algorithm might learn, we have continual models. So the tools got better, but the appetite to use them has increased dramatically, I think, in the last 18 months, because data’s getting very big, very complicated, and no amount of money or time is enough to actually get through it without using this sort of technology.

LN: So are you seeing that other messaging platforms are starting to become more a part of this process, like Slack?

CC: Oh, yeah.

LN: You’ve got all kinds of other messaging platforms, WhatsApp.

CC: Weird data is the new normal and I noticed it starting, I’ve been at Disco about a year, so starting my last 18 months at Gibson Dunn, where it used to be, okay, email, maybe text. That’s all I got to worry about. No, no, no, now I’m dealing with ephemeral messaging, which is self-destructing text messages. I’m dealing with collaboration tools like Slack and Messenger and Teams and each one of these tools has a challenge in terms of formatting the data, being able to review it, and relating it. Think of a given day. This morning, I was on Slack, then I was answering text messages, then I had a phone call, then I sent an email, then I went back to my Slack channel. That was before I got out of bed and if you want to recreate kind of this digital footprint of what people are doing, you need to have all of that info. And so, finding tools and partners that can deal with it is paramount.

LN: So does your platform at Disco, does it have APIs and import specs that match upon those alternate data streams?

CC: We do to a degree. We also do kind of a middleware layer of parsing and creating a new visualization, like say from a JSON file for Slack, we recreate that in our ecosystem and render it the way you would’ve seen it in the Slack dialogue box. And so, we’re developing more of those direct APIs of a 365 box, but we’ve worked on the visualization and ensuring that the data we receive is reviewable, usable, and easily rendered, so.

LN: Now, it’s interesting when we’ve collected cellphone data, we’ve used some of the popular tools on the market and the output of the data isn’t necessarily always easy for the attorneys to review. And what we’ve done is we’ve often taken the spreadsheet output of text.

CC: Oh yeah, yeah.

LN: So what are some of the challenges you see facing AI and its adoption over the next few years?

CC: Like with everything, it’s fear and desire. People desire the outcome of finding stuff faster, being able to practice law, but no attorney went to law school to play with relational databases and lambda calculus. I didn’t. And so, what ends up happening is there’s a fear of the unknown and a fear of explaining something to a judge who maybe didn’t even use a laptop when he was going to law school, probably didn’t. So there is a fear of using technology that folks don’t understand, a fear of explaining it, and that’s when having the right partner, the right person to testify, the right person to navigate you through this becomes so important.

LN: Have you seen much, part of my practice deals with patient electronic medical records?

CC: Oh yeah, yeah.

LN: And patient audit trails of EMR, electronic medical records.

CC: Oh, yeah.

LN: Usually, those records aren’t quite like an email thread. They’re more cryptic. They’re more accustomed to the specific platform the hospital’s use. Have you seen many of those cases come in where they’re pulling in the charts and various transcripts from the physicians and whatnot?

CC: I haven’t run into that as much at Disco, but when I was at PWC, we were doing very complex multilayer investigations, and so, we would have, sometimes, medical charts. Sometimes we would have trade databases and so, marrying and creating a story between that structured data and the unstructured data was always very challenging and very bespoke, and there’s some tech that’s beginning to create a unified place to do that. We’re looking in to do that as well, but it’s very hard to take that weirdly formatted data and render it in a way that then ties to what the humans are saying and then, help you get those facts to build your case.

LN: That’s great. Well, this has been great. In our next segment, we’ll be talking a little bit more about artificial intelligence and some of the potential challenges and impacts for organizations that don’t get on board. So thanks for coming on again.

CC: My pleasure.

View Part 1 of our 3 Part Series on Artificial Intelligence (AI) in the Legal Industry

Re-inventing Legal Technology: Artificial Intelligence (AI)
Part 1 in our Three-Part Series about Artificial Intelligence (AI) in the Legal Industry

View Other related blogs from Enigma Forensics.com

Understanding EMR Audit Trails
Artificial Intelligence (AI) Plays an Important Role in EMR Audit Trails
Hospital Data Breached
Artificial Intelligence (AI) in Hospitals
Energy Industry Incident Response
Artificial Intelligence (AI) in the Energy Sector

View DISCO’s website and receive a free demo

https://www.csdisco.com/

View Law Technology Today LTT as it reviews AI trends in the Legal Industry

AI Trends Driving the Legal Industry

Artificial Technology

Artificial Technology and Medical Data

Enigma Forensics, Lee Neubecker reviews with Eric Fish, the Federation of State Medical Boards, Senior VP of Legal Services, about the positive impact of artificial technology and machine learning on medical standards and regulations. Find answers how this technology will improve the patient experience in the future.

The transcript of the video follows

Lee Neubecker: Hello, I’m here today with Eric Fish, Senior Vice President of legal services. He’s with the Federation of State Medical Boards and he’s going to be talking to us a little bit today about his organization and how they’re using technology to change how things work.

Eric Fish: Thank you, well the Federation State Medical Boards is the organization that represents the 70 state medical and osteopathic boards who are charged by state law to regulate the practice of medicine within the various states, in that we help build standards for regulation, best practices. We also work with states on our data and other things that are exchanged that really help improve the regulation of medicine for the patient, the end user of medicine.

Lee Neubecker: Eric can you tell us a little bit about how artificial intelligence and machine learning are impacting your organization and membership?

Eric Fish: Well, Lee, we’re actually at a, what I believe to be, a crossroads of cultural, social, and technological change that are really going to change the way that we have to look at regulation for the public benefit. There’s going to be a lot more data on patient/provider interactions. There is also going to be much more data consumed by state regulators to see which of these interactions comply with the standards. One of the things that I see developing out of this A.I. and machine learning is that we’re going to be creating much more data that can be mined as a regulator to see what interactions are good and which interactions are bad.

Lee Neubecker: Eric can you tell us a little bit about how A.I. and machine learning are being implemented to improve transparency?

Eric Fish: Well, one of the things that’s going to occur, I believe, is that as patients and providers start turning to algorithms to help with that continuation of care. Really the people who implement these systems have to prove up to the regulators how these comply, how these algorithms, how other things are going to comply with the standards that are there. Artificial intelligence has been in medicine for a long time. Machine learning is a little bit new, where we’re taking some of the discussions and building a knowledge base that’s then going to be applied to the patient experience and regulation isn’t standing in the way of these things. The regulations are there so that they are done the right way and in comply with the standards and being transparent on that beginning end is a really great step toward complying with regulations and making the regulatory process better.

Lee Neubecker: Great, and so, you told me that your organization runs some services that consumers might want to be aware of. What are those and what are they used for?

Eric Fish: Well, one of the things that we do on behalf of our members is collate all the disciplinary and regulatory actions that are taken against a provider, and we have a service called Doc Info, where a member of the public can go look to see if an action has ever been taken against their physician. We have access to all 900,000 plus licensees and their information, and it’s really a great service and use of data that we’ve collated and given out to the public.

Lee Neubecker: Great. Well thanks for coming on today. I know you’ve brought your colleague, Mike Dugan. Who’s going to talk for a little bit. Thanks again for coming to the show.

Eric Fish: Thanks, thank you.

Lee Neubecker: I have Eric’s colleague, Mike Dugan, he’s the CIO of the organization, and Mike can you tell me a little bit more about some of the things that you’re doing to improve the quality of the data and integrity of the information?

Mike Dugan: Sure, surely, thank you. We, in many ways, we are a data aggregator and this involves a credentialing process for physicians so we pull data from national data sources, we pull data from institutions to verify physicians’ identity as well as their credentials, so the training and process that they have done. Historically, these have been very manual processes, but we’ve implemented technology to add additional data sources and also give us flexibility in how we consume data. Historically, it’s been a very structured we need a file in this format and our technology is still evolving, but we’re working it to give us the flexibility to work with any data source available.

Lee Neubecker: What are the concerns that your members have regarding data breaches and the potential complications resulting from them?

Mike Dugan: Well, I think they worry about that quite a bit and if anyone in technology who deals with identity and has information, if you’re not worried about data breaches then you’re missing the point and perhaps should be in another line of work. So, we are given the trust of the physicians and our member boards that when they give us their data that it will be protected and that it will be safeguarded, and we work very hard to do that, proactively. So I think that in this environment and this day and age, that is an activity and a task that we will do, it will never go away. It will be ongoing and we will have to adapt if there is new ways that are found to hack information, we always will have to improve our data security.

Lee Neubecker: Well thanks a bunch for being on the show. I appreciate you taking time.

Mike Dugan: Okay, thank you, thanks for having us.

Read More About Government Privacy Controls on Artificial Technolgy

NIST 800-53: Security & Privacy Controls