One private firm’s artificial-intelligence system is deemed insufficient evidence
ShotSpotter, a gunshot detection firm contracted by police departments nationwide, has recently received criticism for its audio forensics system that, it claims, incorporates “sensors, algorithms, and AI” to identify gunshots and locate their source. While several precincts have praised the company for increasing police response to incidents of gun violence, its accuracy as evidence in court remains questionable.
There are two primary reasons for skepticism: 1) studies have indicated that its algorithm has a propensity for generating false positives, and 2) employees are able to modify the database after alerts come in. Since its system is protected as a trade secret, it has been generally inscrutable from oversight.
As seen in this Associated Press investigation, a State’s Attorney’s Office used ShotSpotter’s data for evidence in a case against a Chicago man. This left him in prison for 11 months before the judge dismissed the case. The report eventually released by ShotSpotter showed that the alert in question was identified differently at first. It alerted to a “firecracker” several blocks away from the alleged scene of the crime — but an employee later revised the identification and location. As a result, prosecutors decided that the “evidence was insufficient to meet [their] burden of proof.”
How could it be improved?
This case emphasizes the importance of accountability in regards to digital evidence on either side of a case. The Health Information Portability and Accountability Act (HIPAA), for example, requires retention of Electronic Medical Records (EMR) stored in Health Information Systems (HIS). Healthcare firms must record a permanent record of all additions, changes and deletions of EMR, including the time and person making those changes.
While ShotSpotter obviously isn’t in healthcare, its system would still benefit from similar transparency. It would help improve the reliability of such information. In this case, such logs would have revealed human intervention earlier on. This would have saved the defendant from the 11 he spent months in prison. In other cases, transparency could support prosecution. Regardless, it would bolster ShotSpotter’s credibility when used as evidence.
It’s possible that we could examine information recorded — when the stored data was originally entered and changes to that stored data — without violating trade secret status to a software provider’s algorithms. HIS software providers have trade secret protection to their software. Still, they are required to disclose all record EMR, as well as the revision history to those records.
Where we can help.
Asking the right questions and gathering all available digital evidence is important to achieving an equitable outcome. Enigma Forensics has experience auditing and authenticating digitally stored electronic evidence. We can assist with validating such claims as genuine.
Understanding EMR Audit Trails is important to any company dealing with (PHI). They must have all the necessary security measures in place and follow them to ensure HIPAA Compliance.
Understanding EMR Audit Trails is essential to a patient’s medical history In medical malpractice litigation. The Health Insurance Portability and Accountability Act (HIPAA) requires that the Electronic Medical Records (EMR) maintain an audit trail including all of the metadata. This EMR audit trail is a piece of highly relevant evidence as to who accessed what in the record, what entries were made and/or changed, by whom and when. Computer Forensic experts are key to effective electronic discovery during medical malpractice litigation.
How do hospitals record, protect, and store data? HIPAA sets the guidelines for the most highly sought after information by the world’s best technology hackers. Medical records are worth 4 times more than credit card information. Managing Personal Healthcare Information (PHI) places Healthcare facilities at risk of cyber attack 24/7, 365 days a year.
Check out this video with Enigma Forensics, President & CEO, Lee Neubecker, and John Blair, a noted Healthcare Industry Cyber Security Expert where they discuss the importance of protecting Personally Identifiable Information (PII).
Lee Neubecker and John Blair
Understanding EMR Audit Trails video transcript follows:
This is the third of the last video in the three-part series on Health Care Industry Cyber Threats: Watch Part 1,Watch Part 2
Lee Neubecker: Hi, I have John Blair, a cyber security expert in the field of healthcare, and John is also involved with understanding patient medical, electronic medical record (EMR) audit trails, so I asked him to come on the show and talk a little bit about that with me. John, thanks for coming back on the show.
John Blair: Thanks, Lee. Glad to be back.
LN: So John, can you tell everyone a little bit about what HIPAA requires of healthcare organizations as it relates to tracking data of caregiving and the patients?
JB: Sure. Most of this is obviously directed at hospitals, but HIPAA also has things called business associates, and any interaction from any entity with, or any user with, PHI is going to be subject to these audit logging. Hospitals use systems called EMRs, so generally those, the audit trails are built into the EMRs by default, but obviously entities can turn those off if they so choose or configure them differently. HIPAA requires that you pretty much log any interaction, whether it’s read-only, view-only, edit, whatever that interaction might be. Identify the user, identify the time, what was done to the record, and that has to be maintained for several years. So it doesn’t matter what a user does with the record. Even if they just view it, that counts as a valid interaction and has to be logged and maintained.
LN: In fact, all of these hospital software systems out there have to be HIPAA compliant, or else the hospitals wouldn’t be able to use the software packages. Isn’t that true?
JB: Right, right. There’s a lot of federal regulations regarding that, that the standards that these systems have to meet in order to get refunds or rebates from the government.
LN: So Medicare funding, reimbursement, obviously is important.
JB: All of that stuff. And audit logs of user activity and interactions, or any interaction with PHI, is a critical component of that.
LN: You know, what I’ve seen is sometimes despite the software packages being EMR, audit trail compliant, that there’s the ability for the software that’s deployed to be altered so that the audit trails aren’t retained as long as required by law.
JB: Yeah, sometimes the storage of the audit logs, it can be overwhelming. So oftentimes they are archived offsite or inappropriate access is given to the audit log itself. And then it possibly can be changed, which ruins the integrity of the log, obviously, and that would be a very bad thing should something come up down the road and you needed that log.
LN: Yeah, and certainly, someone who has the master database administrator password to that back-end system, they could do whatever they wanted.
JB: Yup. But there’s supposed to be logs of that activity, as well, and reviews of those logs, but you’re absolutely right. If you’re an administrator, you can do a lot of damage.
LN: Yeah, I’ve assisted clients before involved in litigation, medical malpractice litigation, with just seeking the truth of what’s there in the records. Most of the time, they think many hospitals are compliant and do have those audit trail records.
JB: Absolutely.
LN: But, they don’t necessarily want to make that data readily available.
JB: No, they don’t. And it depends, it’s a case-by-case scenario, under the advice of counsel and things like that, but it’s very, very sensitive information, and obviously, it’s a public relations nightmare to have a breach of patient data, so they take those things very, very seriously.
LN: Absolutely. So can you tell everyone what PHI stands for?
JB: It’s Protected Health Information, as defined by HHS, there are 18 very specific fields that comprise PHI. PHI is a subset of PII, which is Personally Identifiable Information, but with respect to healthcare, it’s primarily PHI that we’re worried about and those 18 identifiable fields.
LN: Why would hackers want to target health care records?
JB: It’s far more valuable now than several years ago, it was credit card information, basically for year after year. Now, the credit card companies and technology with respect to how quickly a card can be replaced and deactivated. And so, just more money in it to steal medical information. And there’s more flexibility, as well. You can go get drugs, you can do a variety of things, whereas, with the credit card, it’s just money.
LN: If people wanted to launch a targeted scam on individuals, certainly having records that would enable them to filter patients that have Alzheimer’s, might give them an unfair advantage at duping people out of their savings.
JB: Absolutely. Because generally if you get someone’s entire record, you’re getting everything about them: their Soc number, their address, phone numbers, relatives, I mean, all this information is now at your disposal. And loans can be taken out in their names, it’s just a disaster waiting to happen.
LN: So Electronic Medical Records, known as EMR, represent an important target that hackers seek, because of the value of that information, and the uniqueness.
JB: Yup. The price of those records, per record, now varies, but I believe it’s in the $150, $200 range per record if it’s a breach now, and laptops can hold hundreds of thousands of records. So it can be very, very expensive.
LN: But it seems that this is a problem, too, that it isn’t just localized to any one area, it’s universal.
JB: Yeah, it’s across the board. Anyone dealing with PHI has this problem.
LN: How does the cost of a patient medical record compare to a credit card record, compare to the black market?
JB: Yeah, for the last several years, medical records have gained in value every year, while financial records, credit card information have devalued. And it’s to the point now where medical information’s worth four times as much as financial information. And that’s only increasing.
LN: So does that mean that people that work in the healthcare sector in IT and security are going to get paid four times as much as the people of the financial sector?
JB: I wish.
LN: Well, thanks again for being on the show, this was a lot of good stuff. I appreciate this.
An electronic medical record (EMR) audit trail is a log file required by HIPAA of all electronic medical record software systems. The EMR audit trail documents all points of access of a patient electronic medical record system including any actions to modify, view, print or amend the record by replacing or adding new data.
Electronic Medical Record (EMR) Audit Trails are key to effective electronic discovery during medical malpractice litigation. Renowned EMR Computer Forensics Expert, Lee Neubecker interviews Insurance Defense Attorney Bill McVisk who usually helps defend hospitals embroiled in medical malpractice litigation. McVisk discusses common areas of confusion during discovery of patient medical records. Neubecker relays some of his past experiences helping plaintiffs uncover important medical records that are often hidden from plaintiffs during discovery. Enigma Forensics has assisted counsel with conducting depositions relating to Electronic Health Records (EHR) and EMR. The two discuss how electronic medical record systems have often made the process of discovery more difficult and confusing to attorneys and litigants.
The transcript of the interview follows:
The transcript of the interview follows:
Lee Neubecker: Hi. I’m here today with Bill McVisk. He’s a patient medical records expert, a litigator. He works with hospitals that are dealing with EMR-related patient medical records and whatnot. I had him on my show today because I want to talk a little bit about electronic medical records. Bill, they said that electronic medical records were going to revolutionize everything and make everything so much better. What’s the reality of what’s happened since we’ve brought about medical records?
Bill McVisk: A lot of EMR has been great. I mean, there’s an ability of doctors to provide records to other people that they couldn’t have done before. There’s the ability, for instance, of a radiologist to look at a film that was taken, and he can be in San Diego, and the patient can be in New York, and it still works. The problems, though, there are some problems. I mean, the biggest problem I see is that anyone who’s ever gone to a doctor’s… the doctors are focused on their computers instead of focusing on the patient. What they’re doing is hitting all sorts of drop-down menus and stuff, and I think we’re losing something from the standpoint of presenting physicians and nurses in malpractice cases. It creates a situation where you don’t really get a sense of exactly what that nurse or doctor is thinking, and so the records just aren’t quite as helpful in medical malpractice cases as they used to be. On the upside, we can read them now, whereas in the past we had to worry about doctors’ handwriting.
Lee Neubecker: Yeah. I know from experience working as a EMR, a patient medical record expert, that discovery can often become challenging. When an attorney is preparing a witness for deposition related to patient medical records, what are some of the things that you look for and care about in that process?
Bill McVisk: Well, the first thing, quite frankly, is to make sure I have the entire record. I can’t tell you how often I’m getting records where I get part of the record, and for some reason, I don’t know if it’s stored on a different server or what, I’m not getting all of the record. I may get all the physician’s part of the record but not the nurse’s part of the record, and obviously, that’s essential. Other problems, like when I’m preparing a witness for a deposition, the big problem is that they’re not used to seeing these records printed out. I mean, in the past, they would look at the chart, it would be exactly the same as the chart they were looking at in the hospital. Now, they are looking at the chart on a computer screen when they’re in the hospital, but when you’re preparing them for a deposition, you’ve got a paper chart, and the paper chart prints out terribly. Every time there’s a slight change of any kind in the record from one minute to the next, the chart prints out the page again and again and again, so there’s all this stuff, and it’s just getting the nurses and the doctors to know where in the chart their entry is going to be makes it a little bit harder.
Lee Neubecker: Yeah. I have experience working with that, and I know that HIPAA requires that every instance of that medical record, pre-editing and post-editing, that that data be preserved and discoverable, but in reality, a lot of the software packages, they only have reports that run the last version, so to get into the true audit trail, you often have to get into the database backend to get access to that information.
Bill McVisk: Well, and I think audit trails are the other aspect of things that makes it a little bit harder in this situation. In the past, we basically, I could give the original medical record to the plaintiff’s attorney to inspect. If somebody had erased something or done something like that, it’d be pretty obvious. I would hopefully know about it before the plaintiff’s attorney would know about it. Then I’d deal with that. But, it may not be obvious now because people can go in, change records, and now, if an audit trail is suddenly showing me, “Oh, my god, somebody was in and did something “to the record,” and it’s two or three weeks after the treatment was over, or, say, two or three hours after a terrible incident occurred, that’s going to make it look concerning. So I think from our standpoint, it’s a matter of making sure healthcare providers are aware of how to do it in a way that isn’t going to look like you’re trying to fake or lie.
Lee Neubecker: And there’s a big difference between accessing a medical record, and editing it.
Bill McVisk: Right.
Lee Neubecker: That’s where sometimes attorneys on both sides become confused about the significance of what’s happening with the patient record.
Bill McVisk: Right. I mean, records get accessed all the time. Maybe it’s to prepare for a deposition. You have to access the record to look at it. Maybe it’s because there’s followup treatment and you need to access the record. That happens all the time, but sometimes, on these audit trails, it’s not always easy. Is this just an access, or is somebody going in and changing something?
Lee Neubecker: And there’s a whole other layer, too. I know from my experience working with many of the packages that the hospitals often use systems that have something known as sticky notes, where they can put comments about a patient. There’s a wide perception that those notes aren’t discoverable. Just because the software doesn’t have a report that will run it, doesn’t mean that if someone like me is coming in, and I get access to the backend database, those comments about the patient and whatnot become apparent. But unfortunately, it’s difficult to get at that data if you don’t know what you’re looking for.
Bill McVisk: And that creates a real problem if you’re defending the hospital, because if I don’t know about these sticky notes in the beginning, first of all, I’m not going to be thinking, “Oh, my goodness.” Then, if you come and discover them, it obviously is going to be, “Oh. I was trying to hide those notes,” or, “The hospital was trying to hide those notes,” which is always the worst thing you can do as a defendant in litigation. And they’re clearly, if there’s something about a patient in those notes, it’s almost never privileged, it is discoverable, and it should be provided immediately.
Lee Neubecker: Also, you know, there’s a tendency I see for the hospitals to try to cover things up. Do you think that there’s some value in bringing in, when you’re defending a hospital, your own forensic expert to dig around and find out what’s really happening?
Bill McVisk: See, I don’t think the hospitals are intentionally trying to cover stuff up. I really don’t think that’s, I’ve almost never seen that happen. There may be, you know, one or two, but in most of these cases, I think the hospitals are trying to find out what the truth is. That being said, the hospital may not be aware that some of these things, because the risk management for the hospital might not be fully aware of all of the situations that are involved in electronic medical records, and yes, at that point, it may be a good idea for me just to have somebody like you go through those records, let me know. Before I produce them to the plaintiff, I would like to know what’s out there.
Lee Neubecker: It would probably be a lot more useful for you to get just a listing of the changes on the record so you’re not looking at the whole document, but maybe here’s a first instance, and then change one, change two, change three, so you can see before text, after text.
Bill McVisk: Sure.
Lee Neubecker: That’s the type of thing that, unfortunately, there’s not canned reports that are in the software that do that. I think that could be by design of the software makers because they don’t want to make it worse for their clients, the hospitals, but it’s certainly possible that it’s just something that was never asked for.
Bill McVisk: That’s quite possible, and I don’t know any of these software makers, but to me, it would be really helpful to know what those are. Of course, that does make it more discoverable, easily discovered by the plaintiff’s attorneys, but on the other hand, I as a defense attorney need to know about it, and if there’s a change that’s improper, I need to know about it right away.
Lee Neubecker: Yeah. What kind of problems can occur when different providers have different EMR systems?
Bill McVisk: Well, that can create problems of a number of ways. Sometimes, the software of one hospital doesn’t communicate with the software of another. There have been situations, for instance, where a physician enters an order for something to happen, and then because of the software problems, it doesn’t get to the provider who’s supposed to do it, and they don’t know that they’re supposed to do it. That creates serious problems for patient care. And similarly, it’s like, if a hospital is discharging a patient to a nursing home, and they want the nursing home to have a certain specific type of care regimen afterward, that can create problems if they don’t communicate well.
Lee Neubecker: Well, thanks a bunch, Bill, for being on the show. I appreciate it.
