Cross-Platform Emoji Interpretation:Analysis, a Solution, and Applications

Fred Morsta�erUniversity of Southern California

4676 Admiralty Way Ste. 1001Marina Del Rey, California 90292

morsta�@usc.edu

Kai ShuArizona State University

699 S. Mill AveTempe, Arizona 85283

kai.shu@asu.edu

Suhang WangArizona State University

699 S. Mill AveTempe, Arizona 85283

swang187@asu.edu

Huan LiuArizona State University

699 S. Mill AveTempe, Arizona 85283

huan.liu@asu.edu

ABSTRACTMost social media platforms are largely based on text, and userso�en write posts to describe where they are, what they are seeing,and how they are feeling. Because wri�en text lacks the emotionalcues of spoken and face-to-face dialogue, ambiguities are commonin wri�en language. �is problem is exacerbated in the short, in-formal nature of many social media posts. To bypass this issue,a suite of special characters called “emojis,” which are small pic-tograms, are embedded within the text. Many emojis are smalldepictions of facial expressions designed to help disambiguate theemotional meaning of the text. However, a new ambiguity arisesin the way that emojis are rendered. Every platform (Windows,Mac, and Android, to name a few) renders emojis according totheir own style. In fact, it has been shown that some emojis can berendered so di�erently that they look “happy” on some platforms,and “sad” on others. In this work, we use real-world data to verifythe existence of this problem. We verify that the usage of the sameemoji can be signi�cantly di�erent across platforms, with someemojis exhibiting di�erent sentiment polarities on di�erent plat-forms. We propose a solution to identify the intended emoji basedon the platform-speci�c nature of the emoji used by the author ofa social media post. We apply our solution to sentiment analysis, atask that can bene�t from the emoji calibration technique we usein this work. We conduct experiments to evaluate the e�ectivenessof the mapping in this task.

ACM Reference format:Fred Morsta�er, Kai Shu, Suhang Wang, and Huan Liu. 2016. Cross-PlatformEmoji Interpretation:Analysis, a Solution, and Applications. In Proceedings of ACM Conference,Washington, DC, USA, July 2017 (Conference’17), 9 pages.DOI: 10.1145/nnnnnnn.nnnnnnn

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor pro�t or commercial advantage and that copies bear this notice and the full citationon the �rst page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permi�ed. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior speci�c permission and/or afee. Request permissions from permissions@acm.org.Conference’17, Washington, DC, USA© 2016 ACM. 978-x-xxxx-xxxx-x/YY/MM. . .$15.00DOI: 10.1145/nnnnnnn.nnnnnnn

1 INTRODUCTIONSocial media and web communication are a major part of every daylife for most people. Sites like Facebook, Twi�er, and WhatsApp allhave hundreds of millions to billions of users who communicate onthese platforms each and every day. While images and videos havebecome commonplace on these sites, text is still the predominantmethod of communication. �is happens on our smartphones,tablets, and computers billions of times every day.

Communication through text has many key issues that keep itfrom having the depth of face-to-face conversation. One of theseissues is the lack of emotional cues [6]. When conversation iscarried out through text, the lack of non-verbal cues removes keyemotional elements from the conversation [5]. One solution to thisproblem is emoticons, which are combinations of standard keyboardcharacters to create facial representations of human emotion, e.g.,:), :(, ˆ ˆ, and :D. While widely used, there are only a limited numberof character combinations that make a cogent representation of ahuman emotion, and the exact meaning of many emoticons can beambiguous [29]. To provide for richer expression, “emojis” o�er aricher set of non-verbal cues.

Emojis are a set of reserved characters that, when rendered, aresmall pictograms that depict a facial expression, or other object.Unlike emoticons, these are not combinations of characters devisedby the users, but instead single characters that are rendered as smallpictures on the screen. �ere are currently over 1,800 di�erent emo-jis de�ned by the Unicode speci�cation, a number that grows witheach iteration of the speci�cation.1 �ese emojis are either facialexpressions (e.g., “grinning face,” , character code U+1F600),2 orideograms (e.g., “birthday cake,” , character code U+1F382).

While emojis have allowed for increased expression of emotionthrough text, they have an inconsistency. �at is, emojis are ren-dered di�erently on each platform. Just as di�erent fonts display thesame character according to a di�erent style; similarly, each majorplatform has its own font to display these characters. With regularcharacters this is not crucial as each character has a prede�nedmeaning. Emojis do not enjoy this prede�ned de�nition, and these

1h�p://www.unicode.org/Public/emoji/3.0/emoji-data.txt2Unless otherwise noted, when an emoji appears inline, the depiction comes fromEmoji One (h�p://emojione.com/), which is considered the standard.

arX

iv:1

709.

0496

9v1

[cs

.CL

] 1

4 Se

p 20

17

Conference’17, July 2017, Washington, DC, USA Fred Morsta�er, Kai Shu, Suhang Wang, and Huan Liu

changes in rendering can have an impact on the way that the emojiis interpreted. Emojis are o�en small depictions of human faces,so slight variations can make the face look entirely di�erent. �iscan cause a di�erent interpretation of the text than was initiallyintended by the author for emotional interpretation to the text. �isissue was raised in Miller et al. [19], where human workers on acrowdsourcing platform rated the sentiment of emojis. �e resultsof these ratings indicate that the same emoji can be perceived aspositive on some platforms, while it can be perceived as negativeon others. Miller’s �nding is important, with repercussions forthe 2 billion people who use a smartphone.3 Furthermore, recentresearch suggests that emojis are replacing emoticons on socialmedia sites such as Twi�er [23]. With so many people a�ected bythis possibility for miscommunication, it is important that we studythe implications and possible solutions to this problem.

In this paper, we answer the following research questions:RQ1 Does misinterpretation based upon emoji rendering occur in

real world data? Miller et al. [19] discovered this possibilityfor misinterpretation using surveys. We assess if thesephenomena appear in real world social media datasets.

RQ2 What is the scale of this misinterpretation? If this misinter-pretation manifests, that does not necessarily mean thatit a�ects a vast array of communication. We measure theextent to which communication on one social network,Twi�er, is a�ected by misinterpretable emojis.

RQ3 How can the problem of cross-platform emoji interpretationbe addressed? Using our insights from the �rst two ana-lytical portions, we construct a solution that produces amapping of emojis from one platform to those on another.

RQ4 Does correcting for emoji misinterpretation have a mean-ingful e�ect on analysis? We measure the usefulness ofour mapping by applying it to a common text analysistask: sentiment analysis. We show that the performanceis increased by mapping all tweets to a common emojilanguage.

2 RELATEDWORKIn this section we discuss the related work from three di�erentperspectives. First, since our solution heavily relies on word em-beddings to create the emoji mapping, we enumerate some recentwork on word embeddings. Next, we discuss other resources forinformal text and continue to discuss other work that has beendone on emoji analysis. Finally, we discuss other work that hasbeen done in the context of platform-speci�c emoji rendering.

2.1 Word EmbeddingsOne of the �rst word embedding algorithms was the Neural Net-work Language Model [3]. Currently, one of the most famous wordembedding algorithms, Word2Vec [15], has risen to prominence.�is algorithm works embedding words that appear next to eachother in the text next to each other in the embedding. Word2Vecprovides two approaches to solving this problem: the “continu-ous bag of words” (CBOW) and the “skip gram” (SG) architecture.

3h�ps://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/

While skip gram has been shown to be�er solve analogies [16], wecompare with both approaches in this work.

�e simple rules combined with other constraints has beenshown to create powerful embeddings that work in many di�erentse�ings. For example, they have been used to produce state-of-the-art performance in the areas of syntactic parsing [25], namedentity recognition [7], antonym detection [22], sentiment analy-sis [14, 26, 27], and machine translation [31].

2.2 Linguistic Resources for Informal TextBefore emojis were commonplace, there was a long history of tryingto be�er represent and understand text, both formal and informal.One of the most in�uential resources is WordNet [17], which repre-sents words not only by their de�nitions, but also provides a graphof the relationship between the words. Extensions to this approachabound, but perhaps the most relevant one to our work is Senti-WordNet [10], which considers the sentiment of the words whenbuilding the resource. In the context of informal text, SlangSD [30]provides a sentiment resource for mapping slang words to senti-ment scores by leveraging Urban Dictionary’s data. Crowdsourcinghas also been used to extract the emotional meanings for words [20].

2.3 Emoji AnalysisAs emojis have become an important tool that help people com-municate and express their emotions, the study of emojis as theypertain to sentiment classi�cation and text understanding is at-tracting a�ention [1, 2, 9, 11–13, 21]. Hu et al. [12] proposes anunsupervised framework for sentiment classi�cation by incorporat-ing emoticon signals. Hallsmar et al. [11] investigates the feasibilityof an emoji training heuristic for multi-class sentiment analysis onTwi�er with a Multinomial Naive Bayes Classi�er. Eisner et al. [9]learn emoji representation by running skip gram on descriptions ofemojis provided in the Unicode standard. Instead of using Unicodedescription, Barbieri et al. [2] learns the vector skip gram model fortwi�er emojis using tweets, which also demonstrate the ability ofEmojis in improving sentiment analysis. Others analyze the senti-ment of emojis with respect to tweet corpus of di�erent languages,position of emojis in text, etc ( [1, 21]). We build on this work bystudying the di�erence for emoji usage in di�erent platforms.

2.4 Platform Speci�c Emoji Rendering�e aforementioned studies on emoji analysis ignore the fact thatthe same emoji unicode has di�erent emoji images on di�erentplatform. �us, the sentiment or semantic meanings of the sameemoji may be perceived di�erently for people using di�erent plat-forms and thus cause misunderstanding. �erefore, recently, thereare researchers paying a�ention to this issue [19, 28]. Miller etal. [19] show that emoji misinterpretation exists within and acrossplatforms, from both semantic and sentiment perspectives. �eanalysis is based on a survey to collect people’s feedback of senti-ment scores and semantic meaning on di�erent rendering of emojis,which does not consider the context of emojis. Similarly, Tigwell etal. also explore platform-dependent emoji misinterpretation prob-lem in [28]. �ey design a questionnaire to collect user’s sentimentfeedback on 16 emojis from Android and iOS platform, and computea valence-arousal space to guide sentiment analysis. Di�erent from

Cross-Platform Emoji Interpretation:Analysis, a Solution, and Applications Conference’17, July 2017, Washington, DC, USA

Table 1: Amount of data collected by platform. �e“Source(s)” column indicates the applications we chose torepresent each platform.

Platform Source(s) Tweets

Android Twi�er for Android 5,839,392

iOS Twi�er for iPad 12,850,344Twi�er for iPhoneiOS

Twi�er Twi�er Web Client 1,562,655

Windows Twi�er for Windows Phone 114,175Twi�er for Windows

Total 20,366,566

existing approaches exploring platform speci�c emoji renderingproblems, we use real world data to verify the existence of emojiambiguity and provides a mapping-based solution to identify theintended emoji from original posts.

3 PLATFORM-SPECIFIC EMOJI USAGEPrevious work by Miller et al. [18, 19] identi�ed platform-speci�cemoji meaning by carrying out surveys with human participantson Amazon’s Mechanical Turk. While these insights are extremelyuseful, we must verify that these pa�erns truly occur in real-worlddata. In this section we outline our process for collecting an emojidataset and measure the e�ect to which platform plays a role in theuse of emojis.

3.1 A Platform-Speci�c Emoji Dataset�e dataset used in this work consists of social media posts collectedfrom Twi�er. Twi�er is an a�ractive option for our analysis forseveral reasons. First, it is large. With approximately 500 milliontweets each day, it is one of the largest social media sites. Also,because of its 140-character limit, users may be prone to use emojisbecause they can help the user to be more expressive within therestrictive character limit. Furthermore, Twi�er, like many othersocial networking sites, is a place where people post using manydi�erent platforms. Additionally, the site makes the source of thepost available as part of its metadata.

To collect the dataset used in this work, we manually identify asubset of emojis that have human faces or other emotional signals.�e full list of codes used in the data collection will be releasedupon request. Using this list of emojis we query Twi�er’s FilterAPI,4 which takes as input a list of keywords to track, using ourlist of emojis. We tracked this data for 28 days, collecting a total of20 million tweets.

Because the nature of this work is focused on the platform-speci�c nature of the emojis, we separate the tweets based on theplatform from which they were posted. Twi�er is an open platform,meaning that it has a fully documented and available API which4h�ps://dev.twi�er.com/streaming/reference/post/statuses/�lter

Table 2: Average Jaccard coe�cient of emojis across plat-forms. Random indicates a random sample across all tweetsof the size of the iOS corpus.

Android iOS Twi�er Windows RandomAndroid 1.000 0.153 0.111 0.062 0.010

iOS 0.153 1.000 0.086 0.052 0.018Twi�er 0.111 0.086 1.000 0.047 0.005

Windows 0.062 0.052 0.047 1.000 0.002Random 0.010 0.018 0.005 0.002 1.000

any third party can use to make so�ware to post to Twi�er. WhileTwi�er does not explicitly mention which platform the user usedto write the tweet, they do provide details about the so�ware usedto author the tweet in their “source” �eld. �is source is madeavailable in the data that comes from their APIs. In some cases, the“source” is a clear indication of the platform because some so�wareis only available on one platform. We use these when determiningthe platform from which the tweet was posted. We identify fourplatforms with distinct emoji sets according to Emojipedia:5 iOS,Android, Windows, and Twi�er. We select these because they aremajor platforms. �ere are two reasons behind this. First, theresults obtained from these platforms will apply to more userson the social media site. Second, by choosing large platforms wecan accrue a more sizable dataset, which will yield a more stablemapping. Statistics of the dataset, as well as the applications weselected to represent each platform, are shown in Table 1.

Now that we have collected an emoji dataset, we will continue toinvestigate the di�erences between the usage of emojis on di�erentplatforms. Towards answering RQ1, this analysis is performedfrom two perspectives: 1) the positioning of the emojis within aword embedding, and 2) the sentiment of the posts in which theemojis appear.

3.2 Measuring Emoji Embedding AgreementFirst, we investigate how consistent the embeddings of the emojisare across platforms. Word2Vec [15], a word embedding algorithm,learns a vector for each word in the vocabulary. For a given word,the vector is constructed based upon the neighboring words eachtime the given word is used. �us, we employ this technique tomeasure how consistent the usage of emojis is across platforms.

We build a base word embedding by training a skip gram Word2Vecmodel using the entire dataset with emojis removed.6 �en we usethis base embedding to train a platform-speci�c embedding byadding the emojis back in and updating the model with the newdata. It is important to note that we do not update non-emoji words,we only update the vectors of the newly-added emojis. A�er fol-lowing this process, we have 4 platform-speci�c emoji embeddings.To measure how these deviate from general Twi�er conversation,we also create a platform-agnostic word embedding by training arandom sample of tweets of the size of the iOS platform.

Word embeddings have the property that words that are moresemantically similar will be embedded closer together [15], where5h�p://emojipedia.org/6�is process is explained in greater detail in Section 4.1.

Conference’17, July 2017, Washington, DC, USA Fred Morsta�er, Kai Shu, Suhang Wang, and Huan Liu

U+

1F6

28

U+

1F4

75

U+

1F4

82

U+

1F4

4F

U+

1F4

72

U+

1F6

2F

U+

1F6

26

U+

1F4

44

U+

1F4

8F

U+

1F4

48

U+

1F4

71

U+

1F4

69

U+

1F4

4C

U+

1F4

73

U+

1F4

86

U+

1F4

67

U+

1F4

91

U+

1F4

68

U+

1F6

18

U+

1F4

8C

U+

1F4

7F

U+

1F4

74

U+

1F6

1A

U+

27

0C

U+

1F4

50

U+

1F4

6E

U+

1F6

46

U+

1F6

4E

U+

1F4

6B

U+

1F6

B6

U+

1F6

32

U+

1F6

4B

U+

1F6

35

U+

1F6

16

U+

1F6

13

U+

1F6

2E

U+

1F4

49

U+

1F6

45

U+

1F6

0B

U+

1F6

19

U+

1F4

76

U+

27

0A

U+

1F4

98

U+

1F6

2A

U+

1F6

30

U+

1F6

2B

U+

1F6

34

U+

1F4

6A

U+

1F4

70

U+

1F4

77

U+

1F4

93

U+

1F4

85

U+

1F6

1E

U+

1F4

42

U+

1F4

78

U+

1F6

22

U+

1F4

7C

U+

1F4

8B

U+

1F6

37

U+

1F6

27

U+

1F4

46

U+

1F6

24

U+

1F6

2C

U+

1F4

43

U+

1F4

4B

U+

1F6

4F

U+

1F6

06

U+

1F4

97

U+

26

1D

U+

1F6

33

U+

1F6

05

U+

1F6

21

U+

1F6

36

U+

1F6

11

U+

1F4

9A

U+

1F6

0A

U+

1F4

9B

U+

1F4

95

U+

1F6

17

U+

1F6

20

Emoji (Hex Code)

1.0

0.8

0.6

0.4

0.2

0.0

0.2

0.4

0.6

Senti

ment

Emoji Sentiment by Platform

Windows

Android

iOS

Twitter

Figure 1: Average sentiment for each platform by emoji. Error bars indicate the variance calculated from the bootstrappedsamples. A sentiment score of 0.0 is “neutral,” 1.0 is “perfectly positive,” and -1.0 is “perfectly negative.” �e x-axis labelsindicate the Unicode character code of the emoji.

“closeness” is de�ned by cosine similarity. We compare the usage ofthe emojis on each platform by seeing the words that are embeddedclosest to each emoji. For each platform, we extract the closest 1,000words to the emoji. To compare the di�erences across platforms,we compute the Jaccard coe�cient between the top 1,000 on the�rst platform and the top 1,000 on the second. We compute theaverage Jaccard coe�cient across all emojis.

�e results of this experiment are shown in Table 2. �e resultsindicate that the emojis are embedded next to very di�erent wordsacross models. �e most agreeing platforms are iOS and Android,where an average of 153 words are common across the top 1,000 inthe emojis. We also note that Windows has a much lower averageagreement than other models. Finally, all platforms are extremelydi�erent from a random sample. �is means that combining tweetsfrom all platforms, as is done in many analytical tasks, will yield asigni�cantly di�erent representation than considering each plat-form individually.

While we have discovered that the emojis are used in di�erentcontexts across platforms, that does not necessarily mean that theirmeaning is perceived di�erently. To be�er answer this question,we assess the sentiment of the tweets in which the emojis occur.

3.3 Assessing Emoji SentimentHaving collected an emoji dataset, we continue to see if the usage ofthe emojis is di�erent across platforms. To measure the consistencyof the meaning of the emoji, we perform sentiment analysis on thetweets containing the emoji. Using the Pa�ern library’s sentimentanalysis tool,7 we compute the average sentiment for each emoji on

7h�p://www.clips.ua.ac.be/pages/pa�ern

each platform. �is is done by removing the emoji from the tweetand using the sentiment analysis tool to compute the sentimentscore for the remaining text. Finally, we consider the possibilitythat each platform may have a di�erent sentiment “bias,” that is thesentiment expressed on those platforms is di�erent. For example,Windows Phone may be preferred by business users who are lesslikely to express negative sentiment in their posts. To account forthis, we take the average sentiment across all tweets on the platform,and subtract that from the emoji’s score.8 We then take the averagesentiment across all tweets in which the emoji occurs and plot theaverage in Figure 1. Because we only have one corpus for eachplatform, we bootstrap [8] the corpus to obtain con�dence intervals.By sampling with replacement, we create 100 bootstrapped samplesand reproduce the process above to understand the variance in thedata, yielding the con�dence intervals in Figure 1.

�e results of this experiment show several phenomena aboutthe usage of emojis online. First, we see that in many of the emojisthat there is a signi�cant di�erence between the sentiment in atleast two of the platforms. Among those, some even have divergingsentiment polarities. In these cases, if one were to read only thetweets from a particular platform, they would think that emoji has acompletely di�erent meaning than it does on another platform. Toillustrate this point, we provide an example of the emoji di�erences.We include their o�cial de�nition according to the Unicode stan-dard,9 and their depiction across the di�erent platforms in Table 3.Take, for example, the “fearful face” emoji. In the case of this emoji,Figure 1 indicates that Android, iOS, and Twi�er all have this emoji

8All platforms have roughly the same sentiment bias of +0.20 with the exception ofWindows Phone with a sentiment bias of +0.30.9h�p://unicode.org/emoji/charts/full-emoji-list.html

Cross-Platform Emoji Interpretation:Analysis, a Solution, and Applications Conference’17, July 2017, Washington, DC, USA

Table 3: Emojis to illustrate the meaning di�erence. �enames and codes are provided by the Unicode standard.

Name Code Android iOS Twi�er Windows

Fearful Face U+1F628

Clapping Hands U+1F44F

Person Pouting U+1F64E

hovering at the roughly “neutral” area of the sentiment spectrum.Windows, however, is an extreme outlier with most users includingthis emoji in positive tweets. �e intuition behind this phenomenonis demonstrated clearly by the “Fearful Face” emoji shown in Ta-ble 3, where Android, iOS, and Twi�er display the emoji completelydi�erently from Windows. We speculate that the di�erence in theway the emojis forehead is rendered could cause this di�erence ininterpretation. Another di�erence that appears in the “ClappingHands” emoji, this time with Twi�er being the outlier. Android,iOS, and Windows all clearly show two hands with action lines in-dicating that they are moving together. Twi�er, on the other hand,is less perceptible. Only one hand is clearly visible, and this couldgive the impression of a “slap” motion, yielding the more negativesentiment. In the case of the “Person Pouting” emojis, Android andiOS render a signi�cant frown, while Twi�er and Windows have aperson with a slacked mouth.

We have now conducted two experiments on the cross-platformuse of emojis. �ese experiments both illustrate that the use ofemojis is platform speci�c, answering RQ1. In the case of our wordembedding experiment, we �nd that the words that neighbor acertain emoji are vastly di�erent between platforms, indicating thatthey are used in di�erent contexts. �is point is furthered by oursentiment analysis experiment, where we found that some emo-jis have signi�cantly di�erent sentiment scores across platforms,con�rming the results of [19] on a large-scale, real-world dataset.

3.4 �e Scale of MisinterpretationWe continue to address RQ2, which involves measuring the scale atwhich emojis can contribute to miscommunication across platforms.We have used real-world data to show that this problem exists,however, we do not know the extent to which users are a�ected bythis phenomenon. Is this a wide-reaching problem impinging mostTwi�er users or is it an esoteric issue restricted to the few userswho happen to include emojis in their text?

Based on the results of our experiment, we �nd that 38.2% of allemojis yield a statistically signi�cant sentiment di�erence betweendi�erent platforms. While this is a minority, these emojis appear in73.4% of all of all the tweets in our dataset. Since our dataset wascollected using emojis, we need to leverage outside information toestimate the impact for all of Twi�er. To estimate the fraction oftweets using these emojis, we use the Sample API,10 which pro-vides a 1% sample of all of the tweets on Twi�er, irrespective ofwhether they contain an emoji. We collected data from the SampleAPI during the same time period we collected the emoji dataset.10h�ps://dev.twi�er.com/streaming/reference/get/statuses/sample

�rough analyzing this data, we observe that of the 94,233,024tweets we collected from the Sample API during this time period,8,129,483 tweets (8.627%) use emojis that are prone to misinterpre-tation. In other words, 1 in every 11 tweets sent on Twi�er containan emoji that has a statistically signi�cant interpretation to a useron a di�erent platform.

�ese �ndings indicate that there is a need to disambiguateemojis across platforms. To this end, the rest of this paper proposesa strategy for generating a mapping which can disambiguate emojichoice across platforms. Next, we evaluate this mapping to showthat it can increase the performance of sentiment analysis tasks.

4 AN EMOJI MAPPING SOLUTION�e fact that the representations of emojis are di�erent both fromthe perspective of the embedding as well as the sentiment suggeststhat a mapping may help us disambiguate emoji meaning acrossplatforms. In this section, we describe our approach to constructinga platform-dependent emoji mapping, addressing RQ3. �e goalof this mapping is to provide a translation from an emoji on oneplatform to its corresponding emoji on another. �e purpose of theplatform-dependent emoji mapping is to disambiguate platform-speci�c emoji interpretation. To construct the cross-platform emojimapping, we need to understand the semantic meaning and thesentiment polarity of the emojis on each platform. We then canconstruct the platform-dependent mapping by identifying emojisthat have the closest semantic and sentiment polarities across plat-forms. �e key step is to learn the representation of emojis fromthe texts on each platform such that the representations capture theplatform-dependent semantic meanings of emojis and also allowfor similarity matching to construct the mapping.

Recent advances suggest that word embeddings such as skipgram [15] and GloVe [24] are able to capture the semantic meaningsof words. �e low-dimensional vector representations also allowa similarity calculation using cosine or Eulidean distance, whicheases the mapping construction. In addition, recent �ndings onemoji analysis [2, 9] demonstrate that by treating each emoji as aword and performing skip gram on texts, the vector representationlearned by skip gram can capture semantic meanings and sentimentpolarities of emojis, which improve the performance of sentimentclassi�cation.

Word embeddings cannot be directly applied to solve this prob-lem. �ere are two challenges that we must overcome in orderto create the mapping. �e �rst challenge is that we must �nd away to map emojis from a source platform to their true equivalentsemantic emoji on the target platform. �e second challenge is thatwhen we build our platform-speci�c embedding, the position of thewords will change, as well as the emojis. �us, we need to �gureout a way to represent the words within a common space �rst, sothat we can extend it to measure the emojis relative to the words.Our solution addresses both of these challenges.

4.1 Building the EmbeddingFor simplicity, let Tp , p = 1, . . . , P be the set of tweets from eachplatform. �e process of building word and emoji embeddings areillustrated in Figure 2. As shown in Figure 2(a), since the interpreta-tion of words are platform-dependent, for each corpus Tp , we �rst

Conference’17, July 2017, Washington, DC, USA Fred Morsta�er, Kai Shu, Suhang Wang, and Huan Liu

(a) Platform-Independent Word Embedding (b) Platform-Dependent Emoji Embedding

Figure 2: An illustration of platform-dependent emoji mapping construction. �e “Word Embedding” that is output at theend of Figure 2(a) is then used as the “Fixed Embedding” in Figure 2(b). �e three rectangles blue, green, red means corpus ofthree platforms. Each line is a sentence. Triangle and Star denote emojis. Red triangle means an emoji in red platform whileyellow triangle means the same emoji in yellow platform.

remove the emojis, which are denoted as Tp . We then combine Tp ,p = 1, . . . , P , as one large corpus T . �en skip gram is applied onT to learn word embedding W ∈ RK×N , where K is dimensionof the vector representation and N is the size of the vocabularywithout emojis. �e advantages of combining Tp , p = 1, ..., P , asone large corpus T are two-fold: (i) we obtain a large corpus whichallows us to train a be�er word embedding; and (ii) word embed-ding in each platform is the same, which satis�es the assumptionthat word interprations are the same for each platform. A�er theplatform-independent word embedding is learned, then we canlearn the platform-dependent embeddings for emojis. �e processis depicted in Figure 2(b). For each corpus Tp , as the process ofskip gram, we �rst use a context window of size 5 to extract theneighboring words of emojis in the corpus. Let (ei ,w j ) denote apair of neighboring words, where ei means emoji i and w j meansword j. �e extracted pairs are then put into the set Pp . We thenlearn the representation of emoji ei in corpus Tp by optimizing thefollowing problem:

minepi

∑w j :(ei ,w j )∈Pp

(logσ (wT

j epi ) +

N∑k=1

logσ (−wTk e

pi )

)(1)

where epi is the vector representation of ei in corpus Tp . Equation 1is essentially the negative sampling form of the objective functionof the skip gram approach, where we try to learn the emoji repre-sentaion which is able to predict the neighboring words. Note thatthe di�erence with skip gram is that word embeddings W is �xedacross di�erent corpus, we only learn epi . We do the same thingfor each corpus, which gives us epi , p = 1, . . . , P , i.e., the vectorrepresentations of the same emoji in di�erent platforms.

4.2 Constructing the MappingIn this section, we detail the emoji mapping construction process.To construct the mapping between emojis across di�erent platforms,we consider it in a pair-wise scenario. We treat one platform as thesource platform and the other as the target platform. Without loss ofgenerality, let E = {ei , i ∈ {1, ...,m}} be the set of emojis that occurin both platforms. By learning the emoji embedding representationsin each platform, we are able to capture the platform-dependent

semantic features for the emojis. �us, given an emoji in the sourceplatform, we can leverage the emoji embedding representation toconnect the semantic space between the source and target platforms,and then �nd the most similar emoji in the target platform.

Speci�cally, based on all the emoji embeddings from the sourceplatform {esi , i ∈ {1, ...,m}} and target platform {eti , i ∈ {1, ...,m}},we want to map the most similar emoji in target platform for eachemoji in the source platform. To compute the similarity of two emojiembeddings, we adopt the cosine similarity measure as follows,

sim(esi , etj ) =

esi · etj

| |esi | | · | |etj | |

(2)

Given an emoji ei in the source platform, we �rst compute thesimilarity between ei with all emojis in target platform. �en weselect the emoji which gives the maximum similarity score. Wesolve the following objective function to obtain the mapping emojiej in target platform for ei ,

ej = arg maxej ∈E

sim(esi , etj ) (3)

we then get a mapping pair (ei , ej ), where the �rst emoji is fromthe source platform and the second one is from the target platform.Note that the emoji mappings are directional, which means if (ei , ej )is a mapping pair, (ej , ei ) is not necessarily a mapping pair.

5 EVALUATING THE MAPPINGNow that we have presented the methodology for constructingthe emoji mapping, we will validate its utility, answering RQ4.We measure the utility of our mapping by seeing how well it canhelp in a common text analysis task: sentiment analysis. We designexperiments to show that the sentiment analysis task is improved byapplying our emoji mapping to the data to bolster the consistencyof analogy meaning in the dataset. �is simultaneously showsthe e�cacy of our approach as well as the extent to which emojiambiguity plays in standard text analysis tasks.

5.1 Predictive EvaluationWe have shown that emoji usage is di�erent across platforms. �ismeans that the same emoji can have di�erent sentiment meaningsacross platforms. From a machine learning perspective, this means

Cross-Platform Emoji Interpretation:Analysis, a Solution, and Applications Conference’17, July 2017, Washington, DC, USA

that platform-speci�c emoji renderings introduce noise into thedataset, ultimately lowering the classi�cation performance. We hy-pothesize that by applying our mapping we are in e�ect convertingthe dataset into a single language emoji dataset, and the consistentemoji language will help in assigning a sentiment label.

5.2 Experimental SetupIn each experiment, we evaluate our mapping using a pair of plat-forms. One of the platforms is the “source”, and another is the“target.” �e experiment consists of two phases. In the �rst phase,we mix the data from both platforms together, perform sentimentanalysis, and measure the accuracy using 5-fold cross validation.We call the average accuracy across all 5 folds “A1.” In phase two,we apply the mapping to the emojis on the “target” platform, andthen repeat the process in phase one. We call the average accuracyfrom this experiment “A2.” We measure the e�ectiveness of themapping as A2 - A1.

In preprocessing, we remove stopwords, and strip the case fromall words. We tokenize the dataset using the “TweetTokenizer”module in Python’s NLTK [4]. Following the labeling approachoutlined previously, we use the Pa�ern library to assign a sentimentscore to each tweet as ground truth, again hiding the emojis. �esentiment score provided by this library is a continuous value from[-1.0, 1.0]. We convert this problem to a binary classi�cation task.We delete all tweets in the range (-0.2, 0.2) to ignore ambiguouscases. We then assign the sign of the label from the Pa�ern libraryas the label of the tweet. When training the word embeddings aswell as the emoji embeddings, we use K = 20 as the number ofdimensions.

To prevent information leakage between the training and testsets, the emoji mapping used in these experiments is built usingdata collected from September 23rd, 2016 - October 4th, 2016. �etraining and test instances used in our cross validation experimentsare taken from October 5th, 2016 - October 20th, 2016.

Each tweet is represented by the average of the vectors for itswordsW ′ and emojis E ′ in the tweet. We use the target embeddingfor emojis. �us, a tweet is represented as follows:

repr =1|W ′ |

∑wk ∈W′

wk +1|E ′ |

∑ei ∈E′

eti , (4)

where w and et are the source embedding and target embedding.When the mapping is applied, the representation is as follows:

repr map =1|W ′ |

∑wk ∈W′

wk +1|E ′ |

∑ej ∈E′

etj , (5)

where ej is the mapping of the target emoji on the source platform.Having extracted the data, the labels, and formalized the rep-

resentation, we use SVM to build a classi�er using 5-fold crossvalidation. �e only di�erence between each set up is how eachtweet is represented. We compare three tweet representations:

(1) Mapping. �is is the representation where the emojis inthe target platform are mapped to the emojis in the sourceplatform. �is corresponds to the representation obtainedby Equation 5.

(2) NoMapping. �is is the representation when no mappingis applied obtained by Equation 4.

(3) No Emojis. Our hypothesis is that the incorrect emojis areadding noise to our dataset, which hinders classi�cation.Aside from our proposed solution, another way to de-noisethe dataset is to simply remove the emojis. We do this byusing the following representation formula:

repr noemoji =1|W ′ |

∑wk ∈W′

wk . (6)

In this way, emojis are simply not considered in the result-ing feature representation.

5.3 Experimental ResultsWe report the average across the 5 folds in Figure 3. �e resultsoverall are encouraging. In most of the source/target pairs weobtain signi�cantly be�er results than both doing nothing, andremoving the emojis. Further, we note that removing the emojisbeats doing nothing in almost all of the cases, further validating ournoise assumption. �e only two pairs where the mapping does notobtain signi�cantly be�er results are iOS→Windows (“source”→“target”), and Android→Windows. �e lower results across thesetwo cases could be a side e�ect of the nature of the Windows emojis.�e results of our previous analysis indicate that the Windows emojiset is signi�cantly di�erent from the rest in many cases, and thatcould prevent us from learning a quality mapping.

Using the random 1% sample of Twi�er data introduced in Sec-tion 3.4, we discover that 15.0% of all tweets contain at least one ofany emoji. Our analysis from the same section indicates that 8.627%of all tweets, and thus 57.5% of tweets containing an emoji area�ected by this phenomenon. �is justi�es the huge improvementsseen in Figure 3. We speculate that another factor that contributesto the superior performance is the large amount of data upon whichthe embeddings were trained. �ese embeddings are available on-line.11

5.4 Results by Sentiment �resholdIn the previous experiment we removed tweets that had a sentimentscore between (-0.2, 0.2). �e motivation behind this step is thattweets within this threshold were so ambiguous that they could notbe meaningfully assessed for sentiment. In this section, we vary thisparameter to see how robust our method is to ambiguous tweets.We vary the sentiment threshold from 0.1 (leaving many ambiguoustweets) to 0.9 (leaving only the most sentiment-expressive tweets).

Instead of showing the results of every possible combination,which is impossible due to space limitations, we instead test whetherthe two sets of results di�er signi�cantly. �e T-test tests the nullhypothesis that the means of the two distributions are equal. �eresults can be seen in Table 4 for the comparison between Mappingand No Mapping, and in Table 5 for the comparison between Map-ping and No Emoji. In the �rst case we can easily reject this nullhypothesis at the α = 0.05 signi�cance level in all cases except foriOS→Windows. Despite passing the signi�cance bar, this is pairstill yields the least signi�cant result in Table 5. �is is consistentwith our previous result, where this particular mapping did not

11h�p://www.public.asu.edu/∼fmorstat/emojimapping/

Conference’17, July 2017, Washington, DC, USA Fred Morsta�er, Kai Shu, Suhang Wang, and Huan Liu

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

Twi,er->Windows

Twi,er->Android

Twi,er->iOS

Windows->Twi,er

Windows->Android

Windows->iOS

Android->Twi,er

Android->Windows*

Android->iOS

iOS->Twi,er

iOS->Windows*

iOS->Android

Accuracy

MappingPerformance(Accuracy)

Mapping NoMapping NoEmoji

(a) Accuracy

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

Twi,er->Windows

Twi,er->Android

Twi,er->iOS

Windows->Twi,er

Windows->Android

Windows->iOS

Android->Twi,er

Android->Windows*

Android->iOS

iOS->Twi,er

iOS->Windows*

iOS->Android

F1

MappingPerformance(F1)

Mapping NoMapping NoEmoji

(b) F1

Figure 3: Performance results across all “source” X “target” pairs. Asterisks indicate (source, target) pairs where the mappingis not signi�cantly better than the two baselines. F1 is computed with respect to the “positive” class. �ese results are takenwhen deleting tweets with sentiment scores in the range of (-0.2, 0.2).

Table 4: Sentiment threshold signi�cance T-test between“Mapping” and “No Mapping” experimental designs.

Accuracy F1

Twi�er→Windows 3.538e-09 1.200e-05Twi�er→Android 2.493e-08 8.754e-07Twi�er→iOS 8.071e-08 4.887e-07Windows→Twi�er 4.418e-07 5.687e-05Windows→Android 3.538e-09 8.440e-07Windows→iOS 2.115e-08 1.960e-05Android→Twi�er 1.234e-07 1.193e-05Android→Windows 2.586e-07 3.037e-03Android→iOS 2.081e-04 6.212e-06iOS→Twi�er 2.726e-06 1.313e-05iOS→Windows 0.294 0.094iOS→Android 8.877e-08 3.407e-06

fare signi�cantly be�er, and further supports our suspicion thatWindows is an outlier due to the way it renders emojis.

6 CONCLUSIONS AND FUTUREWORKIn this work, we set out to answer a series of questions regarding thenature and extent of cross-platform emoji misinterpretation, andto provide a solution that can help researchers and practitioners toovercome this platform-speci�c inconsistency in their analysis. Inthe introduction, we outlined four research questions based aroundthese issues. Here, we summarize our �ndings as they pertain toeach question and end with a discussion of areas of future work.

RQ1 Does emoji misinterpretation occur in the real world? Toperform this study we use Twi�er, a large, open platform where

Table 5: Sentiment threshold signi�cance T-test between“Mapping” and “No Emoji” experimental designs.

Accuracy F1

Twi�er→Windows 2.466e-06 5.142e-08Twi�er→Android 8.818e-09 1.923e-09Twi�er→iOS 6.121e-08 2.400e-08Windows→Twi�er 6.047e-08 1.665e-07Windows→Android 3.293e-08 3.141e-07Windows→iOS 9.665e-08 1.417e-06Android→Twi�er 3.525e-06 2.021e-06Android→Windows 1.744e-03 1.124e-04Android→iOS 7.423e-08 8.920e-09iOS→Twi�er 2.078e-05 2.954e-06iOS→Windows 3.701e-04 3.399e-02iOS→Android 3.303e-08 4.338e-08

users post hundreds of millions of tweets per day, and where thedata made available by Twi�er contains the source from which thetweet was posted, which can be used to identify the underlyingplatform. We �nd that the sentiment of the tweets in which emojisoccur di�ers widely and signi�cantly across platforms. In manycases, the same emoji exhibited opposing sentiment polarities ondi�erent platforms.

RQ2What is the scale of this misinterpretation? By analyzing arandom sample of tweets, we obtain that 8.627%, or roughly 1 inevery 11 of all of tweets contain an emoji that is used in a statisticallysigni�cantly di�erent fashion on a di�erent platform.

RQ3 How can the problem of cross-platform emoji interpretationbe addressed? With these �ndings, we endeavor to construct anemoji mapping to help researchers, practitioners, and even readers

Cross-Platform Emoji Interpretation:Analysis, a Solution, and Applications Conference’17, July 2017, Washington, DC, USA

of social media data to be�er understand the intended messageof the sender given their platform-speci�c emoji mapping. Weconstruct a solution that exploits the property that “similar wordsare embedded closer together” in order to �nd the correspondingemoji across platforms and build a mapping.

RQ4Does correcting for emoji misinterpretation have ameaningfule�ect on analysis? We evaluate the e�ectiveness of our embeddingby applying it to sentiment analysis. We chose sentiment analysisas it is a prediction problem very common to social media data. Weshow that by mapping all tweets to a consistent emoji vocabulary,we can signi�cantly increase the performance of sentiment analysisby diminishing the amount of emoji noise in the dataset.

�is work opens up the doors for many areas of research. Whilewe have largely explored this platform disagreement in emoji mean-ings from the perspective of computational and predictive tools, itcertainly is not limited to this. In fact, it may be useful to includeour mapping in user interfaces in order to increase understand-ing between individuals communicating on di�erent platforms.Alternatively, social media platforms can take the approach of a“closed” platform, where all platforms conform to a single emoji set.WhatsApp is an exemplar of this closed approach.

�e mapping, and the data used in this work are shared in accor-dance with Twi�er’s data sharing policies online.12 We also provideall of the performance scores used to obtain the signi�cance resultsfrom the sentiment threshold experiment, as well as an expandedversion of Figure 1 containing more emojis.

REFERENCES[1] Francesco Barbieri, German Kruszewski, Francesco Ronzano, and Horacio Sag-

gion. 2016. How Cosmopolitan Are Emojis?: Exploring Emojis Usage and Mean-ing over Di�erent Languages with Distributional Semantics. In Proceedings ofthe 2016 ACM on Multimedia Conference. ACM, 531–535.

[2] Francesco Barbieri, Francesco Ronzano, and Horacio Saggion. 2016. What doesthis Emoji Mean? A Vector Space Skip-Gram Model for Twi�er Emojis. InLanguage Resources and Evaluation conference, LREC, Portoroz, Slovenia.

[3] Yoshua Bengio, Holger Schwenk, Jean-Sebastien Senecal, Frederic Morin, andJean-Luc Gauvain. 2006. Neural probabilistic language models. In Innovations inMachine Learning. Springer, 137–186.

[4] Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processingwith Python. ” O’Reilly Media, Inc.”.

[5] Kristin Byron and David C Baldridge. 2005. Toward a Model of NonverbalCues and Emotion in Email. In Academy of Management Proceedings, Vol. 2005.Academy of Management, B1–B6.

[6] Richard L Da� and Robert H Lengel. 1986. Organizational information require-ments, media richness and structural design. Management science 32, 5 (1986),554–571.

[7] Paramveer Dhillon, Dean P Foster, and Lyle H Ungar. 2011. Multi-view learningof word embeddings via cca. InAdvances in Neural Information Processing Systems.199–207.

[8] Bradley Efron and Robert J Tibshirani. 1994. An Introduction to �e Bootstrap.CRC press.

[9] Ben Eisner, Tim Rocktaschel, Isabelle Augenstein, Matko Bosnjak, and SebastianRiedel. 2016. emoji2vec: Learning Emoji Representations from their Description.arXiv preprint arXiv:1609.08359 (2016).

[10] Andrea Esuli and Fabrizio Sebastiani. 2006. Sentiwordnet: A publicly availablelexical resource for opinion mining. In Proceedings of LREC, Vol. 6. Citeseer,417–422.

[11] Fredrik Hallsmar and Jonas Palm. 2016. Multi-class Sentiment Classi�cation onTwi�er using an Emoji Training Heuristic. Master’s thesis. KTH Royal Instituteof Technology School of Computer Science and Communication.

[12] Xia Hu, Jiliang Tang, Huiji Gao, and Huan Liu. 2013. Unsupervised sentimentanalysis with emotional signals. In Proceedings of the 22nd international conferenceon World Wide Web. ACM, 607–618.

12h�p://www.public.asu.edu/∼fmorstat/emojimapping/

[13] Ryan Kelly and Leon Wa�s. 2015. Characterising the inventive appropriationof emoji as relationally meaningful in mediated close personal relationships.(2015).

[14] Andrew L Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng,and Christopher Po�s. 2011. Learning word vectors for sentiment analysis. InAssociation for Computational Linguistics: Human Language Technologies. 142–150.

[15] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Je� Dean. 2013.Distributed representations of words and phrases and their compositionality. InAdvances in Neural Information Processing Systems. 3111–3119.

[16] Tomas Mikolov, Wen-tau Yih, and Geo�rey Zweig. 2013. Linguistic Regularitiesin Continuous Space Word Representations.. In HLT-NAACL. 746–751.

[17] George A Miller. 1995. WordNet: a lexical database for English. Commun. ACM38, 11 (1995), 39–41.

[18] Hannah Miller, Daniel Kluver, Jacob �ebault-Spieker, Loren Terveen, and BrentHecht. 2017. Understanding Emoji Ambiguity in Context: �e Role of Text inEmoji-Related Miscommunication. In ICWSM.

[19] Hannah Miller, Jacob �ebault-Spieker, Shuo Chang, Isaac Johnson, Loren Ter-veen, and Brent Hecht. 2016. “Blissfully happy” or “ready to �ght”: VaryingInterpretations of Emoji. In ICWSM.

[20] Saif M Mohammad and Peter D Turney. 2013. Crowdsourcing a Word–EmotionAssociation Lexicon. Computational Intelligence 29, 3 (2013), 436–465.

[21] Petra Kralj Novak, Jasmina Smailovic, Borut Sluban, and Igor Mozetic. 2015.Sentiment of emojis. PloS one 10, 12 (2015), e0144296.

[22] Masataka Ono, Makoto Miwa, and Yutaka Sasaki. 2015. Word Embedding-BasedAntonym Detection using �esauri and Distributional Information. In NAACL.984–989.

[23] Umashanthi Pavalanathan and Jacob Eisenstein. 2015. Emoticons vs. emojis onTwi�er: A causal inference approach. arXiv preprint arXiv:1510.08480 (2015).

[24] Je�rey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove:Global Vectors for Word Representation.. In EMNLP, Vol. 14. 1532–43.

[25] Richard Socher, John Bauer, Christopher D Manning, and Andrew Y Ng. 2013.Parsing with compositional vector grammars. In In Proceedings of the ACL con-ference. Citeseer.

[26] Duyu Tang, Furu Wei, Bing Qin, Ting Liu, and Ming Zhou. 2014. Coooolll:A Deep Learning System for Twi�er Sentiment Classi�cation. In InternationalWorkshop on Semantic Evaluation. 208–212.

[27] Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014.Learning sentiment-speci�c word embedding for twi�er sentiment classi�cation.In Association for Computational Linguistics, Vol. 1. 1555–1565.

[28] Garreth W Tigwell and David R Flatla. 2016. Oh that’s what you meant!: reducingemoji misunderstanding. In MobileHCI. ACM, 859–866.

[29] Joseph B Walther and Kyle P D�Addario. 2001. �e impacts of emoticons onmessage interpretation in computer-mediated communication. Social sciencecomputer review 19, 3 (2001), 324–347.

[30] Liang Wu, Fred Morsta�er, and Huan Liu. 2016. SlangSD: Building and Using aSentiment Dictionary of Slang Words for Short-Text Sentiment Classi�cation.arXiv preprint arXiv:1608.05129 (2016).

[31] Will Y Zou, Richard Socher, Daniel M Cer, and Christopher D Manning. 2013.Bilingual Word Embeddings for Phrase-Based Machine Translation.. In EMNLP.1393–1398.