Machine Translation: Analyzing Gender

The Challenge

Machine translation (MT) becomes increasingly important in a global world. Although error rates are still high, MT system accuracies are improving incrementally. Some errors in current systems, however, are based on fundamental technological challenges that require non-incremental solutions. One such problem is related to gender: State-of-the-art translation systems like Google Translate or Systran massively overuse masculine pronouns (he, him) even where the text specifically refers to a woman (Minkov et al., 2007). The result is an unacceptable infidelity of the resulting translations and perpetuation of gender bias.

Method: Analyzing Gender

The reliance on a "masculine default" in modern machine translation systems results from current systems that do not determine the gender of each person mentioned in a text. Instead, the translation is produced by finding all the possible matches for a given phrase in large collections of bilingual texts, and then choosing a match based on factors such as its frequency in large text "corpora" (or bodies of text). Masculine pronouns are over-represented in the large text corpora that modern systems are trained on, resulting in over-use in translations. In July 2012 the Gendered Innovations project convened a workshop to discuss potential solutions. Improving feminine-masculine pronoun balance in these corpora, for example, would still not fix the problem, since it will simply cause both women and men to be randomly referred to with the wrong gender. Instead, it is crucial to develop algorithms that explicitly determine the gender of each person mentioned in text and use this computed gender to inform the translation. Such algorithms could avoid the masculine default and also increase the quality of translation overall.

Gendered Innovations:

1. Studying the Male Default in Machine Translation

2. Detecting the Gender of Entities to Improve Translation Algorithms (research in progress)

3. Introducing Gender-Diverse Translations

4. Integrating Gender Analysis into the Engineering Curriculum

The Challenge

Machine Translation (MT) is an important area of Natural Language Processing (NLP) and a crucial application for an increasingly globalized world. Although machine translation error rates are still high, system accuracies are improving as developers make incremental improvements. Some errors in current systems, however, are based on fundamental technological challenges that require non-incremental solutions. One such problem is gender: State-of-the-art translation systems like Google Translate or Systran massively overuse male pronouns (he, him) even where the text is actually referring to a woman (Minkov et al., 2007). The result is an unacceptable infidelity of the resulting translations and perpetuation of gender bias.

The problem can occur whether translating from English to other languages, or from other languages to English. It is especially common in translating from a weakly gender-inflected language (such as English) to a more strongly gender-inflected language (such as most other Indo-European languages) (Banea et al., 2008). For example, in the English phrase "a defendant was sentenced" it is unclear whether the defendant is a woman or man. When translated into German, this phrase must be rendered in one of two forms—a form specifying a woman defendant or a form specifying a man defendant (Frank et al., 2004):

Humans are able to use the context (the previous or following sentences in the document) to ascertain the gender of the defendant. Current machine translation systems, however, are incapable of using previous or following sentences. The result is that systems simply rely on frequency, choosing whichever gender is more frequently used for this exact phrase—"ein Angeklagter wurde" or "eine Angeklagte wurde"—in the text corpus. Mostly this results in a male default, although sometimes the default is female—see figure below.

Turkish provides another example. The gender-neutral pronoun "o" is translated into English as "he" or "she" in a manner that reflects stereotypical assumptions, such as: "she is a cook," "he is an engineer," "she is a nurse," or "he is a doctor." The same is true for Finnish, Estonian, Hungarian, and Persian (Caliskan et al., 2017, supplement).

Gendered Innovation 1: Studying the Male Default in Machine Translation

Translating from Spanish to English causes a number of gender problems with modern machine translation software. One reason is that Spanish is a pro-drop language. Determining which pronoun to use when translating into English is thus particularly difficult. In March 2011, for example, Londa Schiebinger was interviewed in a Spanish newspaper. Translations into English of the resulting Spanish language article reveal symptoms of the difficulty—see table below.

These automated translations use incorrect pronouns in spite of the many cues to the correct gender in the text, such as:

• "Londa" is a woman's name in English, and is listed as such on online name lists.
• The Spanish source text contains gender-inflected words such as "professor" (profesora, feminine form) and descriptions such as "mujer" (woman) that indicate that "Londa" is a woman.

•      "no le dice nada."
• that literally reads:
•      "doesn't say anything to him/her."
• the Spanish word "le" ("him/her") must mean "her," and not "him."

Method: Analyzing Gender

Machine translation systems have difficulties generating the correct gender for entities because they currently have not solved three basic problems required to make use of the cues to gender that human readers rely on:

• 1. Human translators know that a good translation is one that has the same meaning as the original.
• 2. Human readers recognize when people are being talked about in a text, and represent facts about the people they are reading about, like their gender. They do this even when the people are not mentioned explicitly. For example, a verb that has no pronoun in a pro-drop language needs to be translated with a pronoun in a language like English. A human translator realizes that the phrase:
  •      "'Me encanta el conocimiento,' dice."
  • that literally reads:
  •      "'I like knowledge,' said."
  • is referring to a person who said something, and hence should be translated
  •      "'I like knowledge,' she said."
• 3. Human readers rely on information about which nouns and pronouns corefer in neighboring sentences. For example, the fact that "Londa" is mentioned in the first sentence as the subject of the interview tells a human that the phrase "sus trabajos," or "her/his work," is referring to Londa's work, and hence the translated pronoun should have the same gender as Londa, and appear as "her" and not "his." Human readers know that texts are coherent in this way; they don't just jump randomly from person to person in different sentences. Human translators can recognize coreference in the language they are translating from, and produce coreference in the language they are translating into.

• 1. Our current systems pick a translation, not because it has the same meaning as the original, but because it is most likely to use many of the same words or phrases as a human might use in translating. These are similar but not identical requirements.
• 2. Our current systems do not understand that these sentences are referring to people who have gender. They do not represent gender at all.
• 3. Our current systems do not compute coreference. Current systems are very limited in their ability to use context: They translate a single sentence at a time, and hence are completely unable to use information from previous sentences (like the mention of "Londa") to help in translating following sentences.

MT systems are trained on two separate kinds of text corpora. One is a "parallel corpus" that has a text in one language aligned with its translation in a second language. The other is a large monolingual corpus in just one language that is used to model the grammar of the language being translated into. Both of these types of corpora can lead to errors in pronoun gender. One study on a large monolingual corpus of English, the Google Books corpus showed that masculine pronouns are significantly more frequent in English than feminine pronouns, although this ratio has decreased over time—see chart below. This bias is likely what led to the male pronouns that appeared in the interview example above.

Major organizations now favor or require inclusive language (Rose, 2010). In English, "she or he" is often used when referring to a person whose sex is unknown. Often, recasting a sentence in the plural solves the problem of referents (see Rethinking Language and Visual Representations). As a result, translation algorithms that use "he" as a generic are out of step with modern usage.

Machine translation that does not factor in current language conventions also runs the risk of reinforcing gender inequality. When translation programs default to "he said," the occurrence of the masculine pronoun on the web increases. This may lead to the long-term, unintended consequence of reversing the positive trend of equality in language portrayed on the graph above.

A new review of the literature focuses not only on bias related to women but also non-binary people to overcome the “linguistic visibility” of this group (Kostikova, 2023).

Method: Rethinking Research Priorities and Outcomes

Such algorithms also have the potential to increase the performance of machine translation in people-rich genres like fiction, where current translation systems fail due to their inability to successfully model coherence and coreference (Voigt et al., 2012).

View General Term