Chat content and instant messages: rubbish records on the rise

The chat content challenge

Michael Upton, our Director and Information Management Expert based out of our New Zealand office, takes a deep dive into the challenges of managing chat content for public sector organisations – and how best to address them.

Chat in public sector workplaces has substantially shifted from deskside conversation to something written down. An activity that did not create records now does, almost completely unintentionally. Of course, people still talk to their workmates, but in many public offices there is now a large amount of typing back and forth every day, peppered with links, jokes, emojis, animated GIFs, and references.

Instant messaging tools have been available on many workers’ computers for years, but over the last few years there has been a great wave of increased use. The reasons are obvious: the massive increase in remote working and other flexible arrangements, with the ubiquity of Microsoft Teams – a mix of enabler and catalyst.

When thinking about the benefits of chat, you probably first think of working remotely or across geographic locations, but people are finding the same benefits apply when, for example, someone is in a meeting, off sick, or even on the other side of the floor from you. Sending a quick message to a colleague while you remember is often useful; “side chat” in meetings is often seen as less disruptive than talking over someone, and, even if most of a team is collocated, group chats can help to include those who are not present for whatever reason at that moment.

Chat content and records management

My aim with this article is to explore chat from a records management perspective and to argue that the best available option is to automate the deletion of chat after a fixed period of time. I hope this helps others to firm up their thoughts on how they want to manage these records – whether that is because they agree or disagree with what I am about to write.

I will use “chat” to refer to sending and receiving messages that the recipient can see as good as instantly via software. Both Microsoft Teams and Zoom allow for chat in this sense, but there is nothing in here that, in principle, could not be applied to any other application that serves a similar purpose. Many of the qualities of chat are also shared with emails, but chat is much more resistant to any kind of conventional records management. Microsoft Teams has a concept of channel posts and replies, which I still consider chat for the purpose of this opinion piece. There are some differences which I have chosen to put aside for the sake of keeping this brief.

I will refer to public offices and public records, but I expect anything I write will be equally applicable to local authorities and their records.

There are several qualities of chat that make for poor-quality records, even if we decide some of the content is worth capturing and managing.

A key contributing factor to the poor quality is that almost all chat can be described as unintentional records. The records are not the planned product of a recordkeeping choice, but a byproduct of the combination of the relevant regulatory settings and the medium being used. For example, if a person chooses chat to ask a colleague how to use the printer or what time they are finishing work, it is incredibly unlikely that they made that choice because they wanted to keep a good record.

There certainly will be some occasions when people use chat because they see the value of putting something in writing – or, in other words, making a record. These are usually of short-term value.

So, while chat results in records, it is not designed with good recordkeeping or records management in mind. This is true of the technology and of how people use it.

Here are four factors that would make it hard to identify and manage higher-value chat messages.

1. Information is spread across many separate messages

2. No way to sort the wheat from the chaff

There has been an ongoing trend in written communication towards sending smaller, more numerous messages. In chat, it would not be unusual for someone to convey one idea through a series of five to ten messages, with a sentence or fragment in each. Perhaps an emoji in one, a GIF in another.

It would be very hard and resource-intensive to apply management controls to these many small messages individually, beyond a single blanket rule. For other reasons that I will get to, I am not sure that searching chat content for keywords would ever reliably find messages that need to be retained, but the question of what you would do if you could make this work is complicated by the fact you are dealing with many, small messages. Would you only preserve the specific message which matched the criteria? Or would you keep some messages before and after? How many messages? From which participants? Can you picture one rule that would work in enough situations that you would consider it trustworthy and reliable?

Chat has no meaningful aggregations or descriptive metadata that would allow us to infer connections between the many messages, beyond the order in which messages were sent and to whom. This would make it very hard to meaningfully apply different rules or controls to different subsets of chat messages.

Each message has some very basic technical metadata automatically attributed to it: who it was sent by, when it was sent, and who received it. There are some tools for labelling a chat or communication channel, but it is incredibly unlikely that such descriptions can be used to help with managing records.

3. Forever-changing topics, seriousness, and, in the case of group chats, participants

4. Chat is often only one part of a conversation

If some content of a chat is of value or well-considered, the chat as a whole is at best like a completely unedited meeting transcript that includes all the banter, interruptions and niceties.

A good meeting record does not look like a complete transcript – and, so, does not look like a chat – for good reason. People do not minute the small talk in the room before everyone has arrived or the jokes made when the meeting is interrupted by a fluffy cat, barking dog, or curious child, for example. Minutes intentionally contain a subset of what was said, focused on the information that is likely to have value for future reference.

This fluid nature of chat is another reason that it would be complex and challenging to apply different rules to different subsets of records. It is as good as certain that any valuable messages will be mixed in with many others that are of no ongoing value, even where metadata is available. For example, we cannot reliably say that all chat from one person is more important to retain than another, perhaps because of their role. That one person, if they use chat actively, may write all kinds of things in different contexts, because that is simply how chat works.

The information recorded in a chat may have both missing content and context. Quite often it is only one part of the communication between workers.

A chat could easily pick up again after days or weeks of nothing between participants with a message like:

• “Andy found it”
• “Let me know what you reckon. 🙏” or
• “lol” (the evergreen favourite).

The sender may be following on from a verbal conversation, commenting on some event in the participants’ workday like a meeting or an email, or picking up a thread from a different chat. The instant nature of the medium means participants do not need to explain what Andy found, what they have done or what is making them laugh out loud. A conversation may continue at length without even identifying its subject in writing.

Incomplete content and context raise questions about how much chat content could actually be of value. This factor undermines our ability to reuse the information in chats reliably.

What records and information managers can do with chat

A rule of thumb about contractors

When thinking about what to do about a novel format of records like chat, one thing we can ask is what we would expect of the various kind of third parties that public offices contract to deliver services. The Public Records Act is explicit that a public office is responsible for creating and maintaining records related to matters that have been contracted out.  So, as a rule of thumb, any solution we propose to a records problem needs to work equally well for public office employees and for contractors and service providers of any kind.

It is unsurprising that, when we come up with solutions, we usually think about how we could influence the design of tools and systems within our organisations.  We hopefully have a greater chance of changing how our organisation works than we might expect when contracting services to a third party. However, taking a moment to ask ourselves how a solution would work for contractors can often reveal what matters most and/or a simpler way of achieving a result that is a good fit for the significance of the problem. Of course, we may choose to take different approaches with employees, individual contractors, and other kinds of service providers, but I think we need a clear, compelling justification when doing so.

So, would we talk to a supplier about what they do with their chat records, specifically? Would we expect them to retain chat? Would we expect them to put in place technology solutions for capturing and maintaining access to valuable chat content over years?

To me, some kind of technical solution for capturing chat records feels like the wrong path to go down. The effort required to manage any chat records of value would be substantial and the benefit in doing so marginal. I would be looking for an alternative means to achieve the same outcome that did not require any substantial technology changes.

I think a public office should always give any third party direction on what kind of information must be kept and arrange for how to receive such information, as required. In giving such direction, I would be thinking about the purpose of the Public Records Act that relates to being able to hold the government to account. I might consider contractual terms to direct the client or at least guidance that is relevant to the activity being contracted out. This already aligns with what Archives New Zealand asks public offices to do.

To be specific about how this relates to chat, in most cases I could not imagine even mentioning chat explicitly. If the above contractual arrangements were agreed, I would accept the residual risk that valuable information ended up in the contracted party’s chat messages and nowhere else. That risk feels highly unlikely to come to pass, and it would be even less likely that it would cause a significant problem if it did.

If there was some reason why I thought there was a significant risk that we would not receive the records we need because of something to do with chat, I might work with the contractor to specify those types of information I think are most at risk. I would set the expectation that the substance of those chat messages needs to be documented elsewhere and might be explicit about where and how.

If this is how I think about approaching the recordkeeping expectations for contractors, I need to have good reason to take a different approach for public office staff. Why would I ask them to do something different or configure the public office’s systems to manage the information differently?

But let us consider the available alternatives:

1. We could turn chat off

2. We could manage subsets of chat, according to their value/risk

I cannot picture an office environment where chat will get disabled any time soon. I have read some good arguments for it, but they are as good as moot unless there is another dramatic shift in work culture. The extent to which chat is baked into the incredibly popular Microsoft toolsets would also make this a hard battle to win. And, regardless, you would still need to look at what to do with chat records that are being created now.

For all of the reasons outlined in the previous section, it is not currently practical to identify messages of value, maintain them to a different standard to others, and make them accessible over time. Or, at least, we cannot do this in a way that is more systematic and reliable than having conversations with staff about good recordkeeping, about what makes for a good record, and about where and why to save such a record.

Technical solutions may make it easier to identify some key messages in the future, but I am not clear on what a desirable next step would be.

3. We could keep all chat messages indefinitely

4. We could automatically delete chat of a certain age

Talk to us about your records management needs