Transparency in AI systems is crucial for fostering accountability and trust. It involves sharing information about algorithms, data, and decision-making processes, enabling users to make informed decisions. However, finding a balance between openness and protecting sensitive data and intellectual property is important. Overall, transparency is vital for empowering users empowerment and developing ethical AI as a whole. 

Why does transparency matter?

People often speak about the importance of transparency, especially in AI solutions. But why is transparency so important in building ethical AI? 

Transparency involves publically sharing information about how a system operates. For AI products in particular, transparency means sharing detailed information about how the underlying technology works, which includes the algorithm, training data, data sources, and governance, if any.

Some argue that transparency also involves detailing how an AI algorithm makes decisions, better known as “explainability.” Since AI is a "black box" and we do not yet understand the underlying mechanisms for why AI makes specific choices, this is seen as a limiting factor in building ethical AI. 

The reason why transparency is essential in AI is because it is linked to accountability. Transparency makes system design choices intentional, ultimately, letting the user decide if they would like to continue interacting with the AI. If certain design choices are neglected, users may hold the company accountable and choose a competing product that is in line with their personal ethical stance. 

For example, transparency allows users to know if the programmers intentionally left out training datasets that include certain races. Perhaps the datasets used to train the AI were so large that it was difficult even to know how the AI might behave. But, most importantly, transparency gives the users and company a clear path to solve problems that arise together. In this light, AI is viewed as an evolving product, rather than a flawless one-off solution.

The Origins of Transparency

Organizational Culture 

Transparency as a behavior within an organization emerges from the culture of a company, which almost always originates in the values and behavior of the founder (or co-founders) of a business. A culture that discourages working in isolation and promotes bringing up problems in public encourages open dialogue, candor, and, ultimately, transparency. 

A Culture of Secrecy

Companies often operate on a sliding scale of transparency. On the opposite end of the spectrum, the most convoluted, opaque cultures include a culture of secrecy. People within these organizations don't share information, even with their closest peers. Apple is a good example of a very opaque culture: 

Whether employees are disclosed on certain projects determines which doors their badges can open and which meetings (even which portions of meetings) they can attend. Even within the company, employees were forced to sign project-specific NDAs in “disclosure meetings,” which provide an additional layer of psychological emphasis around the company’s expectations of secrecy.

A culture of withholding information often results in a specific power dynamic within the company, an “unwritten hierarchy of “haves” and “have-nots” within the company. 

Transparent Organizations 

Open-source communities are a good example of the most transparent company cultures. The open-source-software movement supports the use of open-source licenses for some or all software, as part of the broader notion of open collaboration. The culture and history promote a better understanding of code and systems, which are best when executed in the open. 

However, a company open sourcing code does not always entail ethical transparency. An organization may share its code with the public but still be limited by the culture of secrecy. A company may use not wanting to expose their IP as an excuse to avoid accountability by the public.

The Dangers of Openness

Although transparency seems great in theory, there are potential issues with constantly sharing your operations and code in the open. Those against open source argue that "greater transparency exposes a system to potential attacks by revealing an exploitable weakness". By showing your cards, you allow those with malicious intent to identify and take advantage of your weaknesses. 

AI Safety Supporters Against Transparency

One branch of AI Safety supporters is against open-sourcing code and training data, stating that malicious actors might take advantage of the technology and use it to cause large amounts of harm. OpenAI, originally called open due to its intention to remain open to public scrutiny, released GPT-2 at a very limited capacity to limit the potential harm caused by these users. An even more extreme position calls for heavy regulation and banning of open-source AI models and training data in the US to prevent China from catching up with the most recent developments of the technology. 

Those who are pro-open source argue that, eventually, this will happen no matter if open source exists or not. If building is done in the open, then we should be able to, as a public, identify the ethical problems and remove them. Notable organizations who are pro-open source include Meta, HuggingFace, and Mozilla, but to varying degrees - for example, Meta doesn’t share the training data of their Llama models.

Transparency in AI

Balancing Data Privacy and IP

However, full transparency can conflict with data privacy and intellectual property concerns. Imagine completely disclosing the datasets you used to train your AI to the public. This would mean anyone could use them and copy your product and your IP would not be safe.

Nell Watson, in Taming the Machine, emphasizes clarity as the foundation for approaching this dynamic, 

Organizations should maintain transparent records of their AI systems, including design features, data flow, and decomissioning procedures. This information should be tailored to the needs of different stakeholders, from end-users to regulatory bodies. Special care should be taken with sensitive user data, and organizations should be aware of any incentives that might compromise transparency.

How much should users know?

When a user interacts with an AI system, they should be made aware they are dealing with AI and how it functions. User should be informed of their rights and how to control the system, especially if they have special needs or constraints. Also, AI systems should be designed to be open about resource usage. Finally, companies and developers should be open about the potential risks of the system in a way that the users may easily understand.

In practice, let’s imagine a user interacting with a chatbot. The user should be immediately informed that they are interacting with an AI agent. This leaves no room for users to question if they are interacting with a human or an algorithm. While interacting with the chatbot, the user should be able to interact with the bot in a dialogue that answers any potential concerns or clarifies any misunderstandings.