One private firm’s artificial-intelligence system is deemed insufficient evidence
ShotSpotter, a gunshot detection firm contracted by police departments nationwide, has recently received criticism for its audio forensics system that, it claims, incorporates “sensors, algorithms, and AI” to identify gunshots and locate their source. While several precincts have praised the company for increasing police response to incidents of gun violence, its accuracy as evidence in court remains questionable.
There are two primary reasons for skepticism: 1) studies have indicated that its algorithm has a propensity for generating false positives, and 2) employees are able to modify the database after alerts come in. Since its system is protected as a trade secret, it has been generally inscrutable from oversight.
As seen in this Associated Press investigation, a State’s Attorney’s Office used ShotSpotter’s data for evidence in a case against a Chicago man. This left him in prison for 11 months before the judge dismissed the case. The report eventually released by ShotSpotter showed that the alert in question was identified differently at first. It alerted to a “firecracker” several blocks away from the alleged scene of the crime — but an employee later revised the identification and location. As a result, prosecutors decided that the “evidence was insufficient to meet [their] burden of proof.”
How could it be improved?
This case emphasizes the importance of accountability in regards to digital evidence on either side of a case. The Health Information Portability and Accountability Act (HIPAA), for example, requires retention of Electronic Medical Records (EMR) stored in Health Information Systems (HIS). Healthcare firms must record a permanent record of all additions, changes and deletions of EMR, including the time and person making those changes.
While ShotSpotter obviously isn’t in healthcare, its system would still benefit from similar transparency. It would help improve the reliability of such information. In this case, such logs would have revealed human intervention earlier on. This would have saved the defendant from the 11 he spent months in prison. In other cases, transparency could support prosecution. Regardless, it would bolster ShotSpotter’s credibility when used as evidence.
It’s possible that we could examine information recorded — when the stored data was originally entered and changes to that stored data — without violating trade secret status to a software provider’s algorithms. HIS software providers have trade secret protection to their software. Still, they are required to disclose all record EMR, as well as the revision history to those records.
Where we can help.
Asking the right questions and gathering all available digital evidence is important to achieving an equitable outcome. Enigma Forensics has experience auditing and authenticating digitally stored electronic evidence. We can assist with validating such claims as genuine.
EMR Audit Trails as produced by Healthcare Providers during medical malpractice discovery frequently filter out important history of the patient’s medical record. Learn how to compel discovery of the patient’s complete EMR history.
Are you attempting to compel the production of a patient’s electronic medical chart and the complete electronic medical record audit trail?
Medical malpractice litigation today routinely requires obtaining the complete electronic medical record audit trail. Compelling the entire patient’s EMR Audit Trail Discovery is vital to the case. Hospitals, clinics, dentists, and other health providers are required to document patient interactions in electronic HIPAA compliant Healthcare Information Systems (HIS). Electronic Medical Records (EMR) also referred to as Electronic Health Records (EHR) are used almost interchangeably. Requesting and receiving the complete EMR for a harmed party can be a daunting process, especially when health care providers produce voluminous audit trail reports in paper form that lack any clear documentation of exactly what changes were made to the EMR.
HIPAA compliant HIS software providers are required to log all access, review, editing, and deletion of records. Such logs must include a record of the user making the change, the source computer that made the change, the date and time of the records actual creation (this can be different than the date and time stamp that appears on the printed patient chart or progress notes), and all versions of the chart as it existed at various points in time. While the HIS software providers maintain HIPAA compliance, ensuring that deleted or revised patient records remain in the HIS record, those earlier revision instances or deleted (marked inactive) records are routinely left off the patient’s printed EMR. By design, the EMR audit trail reports lack the specific modifications being made and by whom. It is often necessary to formulate your discovery request in a specific way to ensure that all audit trail logs from all of the various HIS-connected systems are produced in such a way that provides a clear understanding of health care events that took place.
The following graphic depicts the typical process involved with retaining a computer forensics expert skilled in deciphering EMR to assist with compelling discovery of the complete patient electronic medical records, including the revision history.
1. Request Patient’s Complete Electronic Medical Records (EMR)
It is important that your discovery request includes important relevant details and enough specificity to ensure you receive a comprehensive production of available information without having unnecessary filters applied. We have seen routine usage of filters such as named users, narrow start and ending dates, departments and other available filters that result in receiving an incomplete production of the patient’s EMR. If you would like a sample electronic medical record discovery request list of items, please call us and we would be happy to share our sample request with you. Engaging our firm early on in the process can help speed things along.
2. Review Produced EMR Records
Reviewing the timeline of events and the complaint to develop an understanding of the critical moments when decisions were made or not made leading to harm to the patient is usually the starting point for engaging a computer forensics expert to assist you. Following the review of the case documents, converting the EMR produced to a more usable format is important before analysis begins. Ensuring that the EMR has been OCR’s, adding page labels to the document if missing saves time downstream and allows for surgical review of voluminous EMR to isolate records of care by date, time, health care provider name, medication, or other activity. Summarizing data and performing focused reviews around key dates and times can provide important insights.
3. Identify examples of withheld records or apparent manipulation
During the review process, it is helpful to identify examples of abnormalities or notations that indicate other data referenced is not contained in the production of the patient’s EMR. Reviewing the complete EMR records produced, not just the critical dates and times, can often help establish normal patterns of EMR and can be used in contrast to critical dates and times where EMR appears to be missing from the record. Skilled and experienced EMR data forensics experts often find indicators of manipulation that may not be readily apparent to someone who is not an EMR data forensics expert. Plaintiff’s medical malpractice counsel should send a written or emailed request to the health care provider to produce apparently missing records. This documentation of asking for the missing data will be helpful later when a motion to compel is filed with the court. Judges always like it when litigants attempt to work things out first amongst themselves before seeking judicial intervention. It is not uncommon that our firm is retained at this stage when the non-expert has reviewed the EMR produced and suspects something is hinky. Having your EMR data forensic expert assist with drafting the follow-on request for missing EMR can help lay the foundation for a later affidavit in support of a motion to compel.
4. Review Supplemental Production of Records if Received
In many cases, healthcare providers will partially respond to a supplemental request for EMR. The production oftentimes still lacks the clear ability to correlate the revision history of the patient’s chart and medical record. The review of all of the EMR produced to date is important in beginning to build the argument to be included in the future EMR expert witness affidavit in support of an onsite inspection of the HIS to obtain the patient’s complete EMR including the revision history.
5. Affidavit in Support of Motion to Compel Onsite Direct Inspection
The EMR data forensics expert must lay the foundation documenting their credentials, what they reviewed, significant findings, notes of any deficiencies in the production, and establishing that additional information not produced by the health care provider may be available from performing an onsite inspection. Direct engagement with the HIS can often reveal additional details such as the actual time or original entry of a notation as well as the life cycle of modification over time showing which device was used to access or modify the notation, what user accessed/modified the record, and the current status of records entered into the EMR. Inactive or deleted notations may be revealed on some HIS systems by toggling the view settings to show inactive records. The sworn statement by the EMR data forensics expert is an important tool in winning your motion to compel and often is filed with the motion, or submitted shortly after and before the hearing on the motion. In some cases, sharing the EMR data forensics expert’s curriculum vitae with the health care provider and the signed affidavit in support of the motion to compel onsite recorded inspection of the patient’s EMR may result in an agreement to allow inspection without the court’s order or an acceptable settlement offer. It never hurts to try.
6. File Motion to Compel Onsite Direct Inspection of the EMR System
Usually, to obtain direct onsite inspection of the healthcare provider’s HIS is a request likely to encounter objections and resistance. Filing a motion to compel and providing a supporting EMR expert witness affidavit can help overcome objections. A federal U.S. District Court ordered a hospital to provide such direct access to a patient plaintiff in a medical malpractice case. (Borum v. Smith, W.D. Ky. No. 4:17-cv-17, 2017 U.S. Dist. LEXIS 109249 (July 14, 2017)). The court’s decision and arguments can be viewed at this link. Onsite inspections can also be performed using remote control/viewing software such as WebEx, Zoom, TeamViewer, and others if the court allows and so orders. Typically, healthcare provider staff or HIS software consultants with administrative access to the HIS will perform the actions directed by the plaintiff’s EMR consultant and allow for recording screenshots of the patient’s EMR as viewed within the software.
7. Court Testimony in Support of Motion to Compel Onsite Direct Inspection
Having your EMR expert present in the hearing on your motion to compel usually takes place in person or via a remote video conferencing tool such as Zoom. Since the outbreak of Covid-19 began to escalate in 2020, courts have become more comfortable with allowing remote experts to appear via electronic video conferencing, making it easier to retain the most knowledgeable EMR computer forensics expert witness without concerns over the geographic location of your expert witness. Allowing the judge to ask questions of your EMR expert witness directly and assist you with responding to any raised objections has been proven to be highly effective in winning the motion to compel onsite inspection of the plaintiff’s EMR.
8. Onsite Inspection
Once the court has granted the motion to compel an onsite inspection, it is important to ensure that any in-person meeting isn’t a waste of everyone’s time. Problems that can arise include the health care provider producing someone to operate the computer terminal who is not knowledgeable about how to use the HIS or that lacks full administrative access to the complete backend databases containing detailed historical information including revision history of the EMR. In some cases, such as Cerner and Epic, some screens can be viewed in the software that will show progress notes and the revision histories including the user name modifying or entering the record and the times the record was updated by the user. In other systems, it may be necessary to access the back-end database system with administrative credentials to perform Structured Query Language (SQL) queries to identify the relevant record history. Having an EMR expert that has experience writing SQL database queries is important when the HIS doesn’t offer a built-in report or display view that can show the complete historical record of events.
9. Review Records Captured Onsite
Following the onsite inspection, it is often necessary to review in more detail the screenshots and video footage documenting the EMR in the HIS. Reports generated during the onsite may need to be compared against earlier productions of EMR to help document any records that were withheld. Where it is provable that the healthcare provider withheld patient EMR, it may be possible to petition the court to order reimbursement of expert witness fees associated with the consulting engagement.
10. Write Final Report
Many times, a final report is not necessary. Typically, once it is established that records were withheld, or it is believed to be known that this may be the case, it is more often than not that a settlement offer is made to the plaintiff when obfuscation or manipulation of the patient’s EMR took place. If no acceptable settlement is reached, writing a final report in the form of a sworn affidavit to detail the delays and extra costs associated with discovery is important for petitioning the court to award expert fees. Other times, the data obtained from the onsite inspection can be presented without a report or sworn affidavit. Photos and videos can sometimes avoid the need to generate a final report.
11. Expert Witness Deposed
Should an acceptable settlement offer not have been reached, the EMR expert witness will be deposed. This typically is preceded by a request for the disclosed expert witness’s communications with counsel and any work product or notes. Working with an EMR expert witness that has been deposed numerous times and has achieved successful outcomes following the given deposition can make or break your case. If the defense counsel can undermine the credibility of your expert, the admissibility of any of the opinions sworn to by your expert may be excluded. If your EMR expert witness is successful at establishing that records were held back or manipulated and provides a reliable deposition in support of those opinions, your case matter is likely to receive a reasonable settlement offer proportionate to the offenses and harm caused to your client.
12. Trial Testimony
It is rare that you will need your EMR Expert Witness to testify at trial regarding manipulation or withholding of evidence. If the facts exist and have been produced, they often speak for themselves. Many healthcare organizations face frequent malpractice litigation. If it is established in the public record that a healthcare organization permanently deleted a patient’s EMR, that organization could lose Medicare/Medicaid funding for not maintaining HIPAA compliance, a problem that could far exceed paying out a settlement to a single aggrieved party.
13. Case Settles
Medical malpractice cases often settle when it has been established that the records have been altered to distort the true record of patient care. Having news reports published detailing how a healthcare organization manipulated historical patient EMR to mask a mistake resulting in the harm of the patient would only invite more litigation by other harmed patients. In the interest of protecting their organization from further litigation and more intrusive discovery, healthcare organizations need to maintain their profitability and minimize costs paid out for ongoing litigation.
When you are getting stonewalled by a healthcare organization and feel that you are receiving cryptic EMR audit trails, or a production that is missing data that should exist, having an experience EMR computer forensics expert witness and consultant on your side can help you achieve a better outcome for your client. If you would like to discuss a case matter with us, we are happy to provide a complimentary consultation. Call us today at 312-668-0333.