Post on 09-Aug-2020
Handbook of Quantitative Science and Technology Research
1
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
1 Scholarly Twitter metrics
Stefanie Haustein
stefanie.haustein@uottawa.ca 1 School for Information Studies, University of Ottawa, Ottawa, (Canada)
2 Centre interuniversitaire de recherche sur la science et la technologie (CIRST),
Université du Québec à Montréal, Montréal (Canada)
Abstract Twitter has arguably been the most popular among the data sources that form the basis of so-called
altmetrics. Tweets to scholarly documents have been heralded as both early indicators of citations as
well as measures of societal impact. This chapter provides an overview of Twitter activity as the basis
for scholarly metrics from a critical point of view and equally describes the potential and limitations
of scholarly Twitter metrics. By reviewing the literature on Twitter in scholarly communication and
analyzing 24 million tweets linking to scholarly documents, it aims to provide a basic understanding
of what tweets can and cannot measure in the context of research evaluation. Going beyond the limited
explanatory power of low correlations between tweets and citations, this chapter considers what types
of scholarly documents are popular on Twitter, and how, when and by whom they are diffused in order
to understand what tweets to scholarly documents measure. Although this chapter is not able to solve
the problems associated with the creation of meaningful metrics from social media, it highlights
particular issues and aims to provide the basis for advanced scholarly Twitter metrics.
1.1 Introduction Enabled by the digital revolution, the open access and open science movement, big data and the success
of social media have shaken up the scholarly metrics landscape. Academic careers are no longer shaped
only by peer-reviewed papers, citation impact and impact factors, university managers and funders
now also want to know how researchers perform on social media and how much their work has
impacted society at large.
Bibliometricians have started to adapt to the policy pull and technology push and expanded their
repertoire of scholarly metrics to capture output and impact beyond the ivory tower, so far that some
speak about a scientometric revolution (Bornmann, 2016). Metrics are no longer restricted to formal
parts of communication but expand beyond the borders of the scholarly community (Cronin, 2013a).
Similarly to how the Science Citation Index formed the field of bibliometric research and research
evaluation, the altmetrics, or more precisely the social-media metrics landscape is being heavily
shaped―if not entirely driven―by the availability of data, in particularly via Automated Programming
Interfaces (APIs) (Haustein, 2016; Priem, 2014).
Twitter has arguably been at the epicenter of the earthquake that has shaken up the scholarly metrics
landscape. The majority of altmetrics research has either focused on or included Twitter (see Sugimoto,
Work, Larivière, & Haustein (2017) for a review of the literature). Following a general definition of
scholarly metrics which include activity on social media (Haustein, 2016), scholarly Twitter metrics
are defined as “indicators based on recorded events of acts [on Twitter] related to scholarly documents
[…] or scholarly agents […]” (Haustein, 2016, p. 416).
Although findings of an early study had suggested that tweets were a good early indicator of citations
for papers published in the Journal of Medical Internet Research (Eysenbach, 2011), the
generalizability of this claim was refuted by low correlations reported by more representative studies
(Costas, Zahedi, & Wouters, 2015a; Haustein, Peters, Sugimoto, Thelwall, & Larivière, 2014). Low
Handbook of Quantitative Science and Technology Research
2
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
correlations between tweets and citations did, however, spark hopes that Twitter activity was able to
reflect impact on users and use beyond citing authorsa new type of previously unmeasurable impact,
possibly on society at large. Twitter’s popularity in the altmetrics realm has essentially been caused by
two factors, which are both heavily influenced by technology and the data push and policy pull
described above:
a) a significant number of scholarly articles are shared on Twitter, producing a measurable signal;
b) Twitter is a social media platform created for and used by a wide and general user base, which
theoretically has the potential to measure impact on society at large.
As tweets represent an “unprecedented opportunity to study human communication and social
networks” (G. Miller, 2011, p. 1814), Twitter is being used to analyze a variety of social phenomena.
Centering on either the message (i.e., the tweet, its content and associated metadata) or social
connections (i.e., the network of follower-followee relations), tweets have been used to show
discussions during upcoming elections, how people communicate during natural disasters, political
upheaval, cultural events and conferences and have even been used to predict elections outcomes and
the stock market (Rogers, 2014; Weller, Bruns, Burgess, Mahrt, & Puschmann, 2014).
While only a small share of academics use Twitter for scholarly communication (Rowlands, Nicholas,
Russell, Canty, & Watkinson, 2011; Van Noorden, 2014) and to diffuse scientific publications (Priem
& Costello, 2010), more than one fifth of recent journal articles are being tweeted (Haustein, Costas,
& Larivière, 2015), which adumbrates that it is non-academics who engage with scholarly publications
on Twitter. At this point, social media-based indicators have flourished rather as vanity
measuresculminating in a tongue-in-cheek metric called the Kardashian Index (Hall, 2014)than
as validated indicators of societal impact. Even though altmetrics have left their mark on the scholarly
publishing and metrics landscape (Desrochers et al., 2018), they have not (yet) established themselves
within the reward system of science, where citations remain the only hard currency:
Neither Twitter mentions nor Facebook ‘likes’ are, for now at any rate, accepted currencies in
the academic marketplace; you are not going to get promoted for having been liked a lot, though
it may well boost your ego. (Cronin, 2013b, p. 1523)
Still, almost all big publishing houses now report some form of article-level metric based on social
media activity, including tweets. Despite the lack of validation and a clear definition regarding the type
of impact measured, the number of tweets are thus already used as scholarly metrics “in the wild”
(Priem, Piwowar, & Hemminger, 2012).
This chapter aims to contribute to the understanding of Twitter and Twitter-based metrics with a
particular focus on their potential and limitations when applied as scholarly metrics. To provide some
context for the meaning of scholarly impact measures derived from tweets and Twitter activity, this
chapter describes Twitter’s role in scholarly communication. It depicts how Twitter is used in academia
and how scholarly contents are diffused and discussed in tweets. The chapter provides an overview of
the literature of Twitter use by the scholarly community and scholarly output on Twitter. The latter
part is supported by empirical results based on an analysis of 24 million tweets mentioning scientific
papers captured by the data provider Altmetric.com. Both the review of the relevant literature and the
patterns extracted from the Twitter data are intended to contribute to the understanding of what type
of scholarly contents are diffused on Twitter, who is diffusing them, when and how. This will help to
assess Twitter metrics as valid impact indicators and to interpret their meaning.
Handbook of Quantitative Science and Technology Research
3
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
1.2 Twitter in scholarly communication Twitter launched in 2006 as a public instant messaging service and evolved from an urban lifestyle
social network, where users would update their friends about what they were doing, to a platform for
communicating news and events used by 500 million users worldwide, or 23% of US adults online
(Duggan, Ellison, Lampe, Lenhart, & Madden, 2015). Although other microblogging platforms (e.g.,
Sina Weibo, tumblr, Plurk) exist, tweeting has become a synonym (and preferred term) for
microblogging and Twitter the most popular service. Twitter constraints microposts to a maximum
length of 140 characters, a restriction that originates from the 160 character limit of text messages.
Users can follow each other and create user lists to manage the updates they receive from other Twitter
users. Similarly to regular blogs, microblogs are ordered sequentially in reverse chronological order
and, due to their brevity, usually appear more frequently (Puschmann, 2014), making Twitter the “most
dynamic and concise form of information exchange on social media” (Grajales, Sheps, Ho, Novak-
Lauscher, & Eysenbach, 2014, p. 5). While the brevity of tweets is seen as a restriction by some, others
perceive it as a particular advantage:
The brevity of messages allows [tweets] to be produced, consumed, and shared without a
significant amount of effort, allowing a fastpaced conversational environment to emerge. (boyd,
Golder, & Lotan, 2010, p. 10)
Tweets have three major specific affordances, which facilitate communication on the platform:
retweets (RTs), user mentions (@mentions) and hashtags (keywords following #). All of these
functions originated within the Twitter user base and were eventually adopted by Twitter, representing
a co-creation of functions by users and developers. Twitter provides three main levels of
communication: interpersonal communication on the micro level, meso-level exchanges of people who
are directly connected through their network of followers and followees, and hashtag-centered macro-
level communication which enables exchanges among all Twitter users with common interests (Bruns
& Moe, 2014).
In academia Twitter is used to disseminate and discuss scholarly outputs and other relevant
information; maintain collaborations or find new ones; as a virtual “water cooler” (Veletsianos, 2012,
p. 347) for social networking with colleagues; to increase student participation in teaching; as a
backchannel at scientific conferences to foster discussions among conference attendees and those who
participate remotely; as well as to increase visibility and reach wider audiences (Nentwich, 2011;
Osterrieder, 2013; Pearce, Weller, Scanlon, & Ashleigh, 2010; Van Noorden, 2014; Veletsianos, 2012;
Zhao & Rosson, 2009).
1.2.1 Twitter uptake
Twitter is used by various stakeholders in the scholarly community, including individual researchers
and academics, journals and publishers, universities and other academic institutions, as well as at
scholarly conferences. From the perspective of using Twitter activity as the basis for scholarly metrics,
it is essential to know Twitter uptake in academia, as it informs about biases and differences between
disciplines and other user demographics, which may have a direct effect on derived metrics.
1.2.1.1 Scholars on Twitter In the scholarly context, Twitter use by academics lags behind its uptake among the general public.
Although the majority of researchers are aware of the platform, most do not make use of it in a
professional context, giving it the reputation of a hype medium in academia (Carpenter, Wetheridge,
Tanner, & Smith, 2012; Gu & Widén-Wulff, 2011; Pscheida, Albrecht, Herbst, Minet, & Köhler, 2013;
Van Noorden, 2014). A certain reluctance in academia to use Twitter might be caused by its perception
as a shallow medium that is used to communicate “pointless babble” (Kelly, 2009, p. 5) rather than
informative content (Rogers, 2014). Described as “phatic” (V. Miller, 2008, p. 396), Twitter is less
about what people tweet rather than how they are connected.
Handbook of Quantitative Science and Technology Research
4
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Reported Twitter uptake shows extreme variations depending on user demographics―in particular
disciplinary orientation―and when a study was conducted; it usually stays behind use of other social
media. For example, a survey among 2,414 researchers conducted in 2010 demonstrated that while
more than three quarters used social media, less than one fifth were on Twitter (Rowlands et al., 2011).
A more recent study conducted by Nature also showed that Twitter was among the least used social
media platforms in academia: while almost half of 3,027 science and engineering researchers regularly
used ResearchGate, only 13% regularly visited Twitter (Van Noorden, 2014). At almost one quarter
of regular users, Twitter uptake was higher among the 482 social science and humanities scholars
participating in the same survey (Van Noorden, 2014).
Depending on the sample and when the survey was conducted, Twitter uptake varied heavily between
a few percent to more than one third of surveyed scholars using Twitter, which calls the
representativeness of findings into question. Moreover, surveys vary in terms of whether or not they
differentiate between general Twitter uptake, Twitter use for scholarly communication and
professional purposes or active vs. passive use, which further complicates comparison and
generalization of findings regarding Twitter uptake in academia. Twitter represented the social media
tool with the highest difference between awareness and use. Although known by 97% of university
staff in Germany, as few as 15% used Twitter and 10% used in a professional context (Pscheida et al.,
2013). A similar use-to-awareness ratio was found by other studies, for example, at Finnish universities
(Gu & Widén-Wulff, 2011) or researchers surveyed by Nature (Van Noorden, 2014), and academic
staff in Germany (Weller, Dornstädter, Freimanis, Klein, & Perez, 2010).
Most studies found self-reported Twitter use in academia at around 15%; an uptake of 13-16% was
reported for surveys based on 215 health services and policy researchers (Grande et al., 2014), 454
geographers (Wilson & Starkweather, 2014), 1,058 UK academic staff (Tenopir, Volentine, & King,
2013) and 3,027 scientists and engineers (Van Noorden, 2014), while uptake was lower (7-10%) for
academics in Germany (Weller et al., 2010) and the UK (Procter et al., 2010; Rowlands et al., 2011).
Although 18% used it, Twitter was the least popular social media tool of 345 European scholars (Ponte
& Simon, 2011). The highest Twitter use was reported for a survey of 126 Finnish university staff at
23% (Gu & Widén-Wulff, 2011), 1,910 professors at US universities at 32%23% for professional
purposes(Bowman, 2015b), 382 urologists attending a conference at 36% (Loeb et al., 2014) and 71
participants of the 2010 Science & Technology Indicators (STI) conference in Leiden at 44%
(Haustein, Peters, Bar-Ilan, et al., 2014).
As an alternative to determining Twitter uptake through self-reported use in surveys, studies have also
assessed the extent of scholarly microblogging based on Twitter activity of scholars. Identifying
scholars on Twitter is challenging, as the 160-character Twitter bio and the provided user name are
often the only basis for identification. Most studies thus search for Twitter users based on a list of
names of academics (Darling, Shiffman, Côté, & Drew, 2013; Desai et al., 2012; Desai, Patwardhan,
& Coore, 2014; Priem, Costello, & Dzuba, 2012; Work, Haustein, Bowman, & Larivière, 2015) or
apply snowball sampling starting from a set of known scholars on Twitter (Chretien, Azar, & Kind,
2011; Holmberg, Bowman, Haustein, & Peters, 2014; Holmberg & Thelwall, 2014). Searching for a
list of 8,038 US and UK university staff, Priem, Costello, and Dzuba (2012) found Twitter accounts
for 2.5% of them. Although the authors admit that their study underestimated Twitter use, it reflected
that the microblogging platform is not popular in academia and thus confirms finding by most surveys.
Investigating Twitter use in the scientometric community, Bar-Ilan et al. (2012) found Twitter profiles
for 9 of the 57 presenters at the 2010 STI conference.
Other than searching for known scholars on Twitter, some studies try to extract information from
Twitter to identify scholarly users. The most common approach is to classify users based on searching
for specific words in the Twitter bio. Retrieving users whose Twitter bio contained words such as
Handbook of Quantitative Science and Technology Research
5
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
university, PhD or professor, Barthel et al. (2015) identified scientists’ Twitter accounts with a
precision of 88%. False positives contained university accounts or those of non-academic staff at
research institutions. Recall cannot be determined in such studies as the number of false negatives, that
is the scientists on Twitter who do not include any of the queried keywords in their self-descriptions,
remains unknown.
Altmetric also applies a keyword-based approach to categorize Twitter users as scientists, science
communicators (journalists, bloggers, editors), practitioners (doctors, other healthcare professionals)
and members of the general public. It should be noted that Altmetric’s general public category includes
all users that can not be classified as belonging to any of the other three groups and is therefore not a
good indicator of how much an article has been tweeted by members of the general public. An obvious
limitation of the keyword approach is that it is unable to capture scholars who do not identify
themselves as such or who do not use the terminology or language covered by the list of keywords.
However, many scholars seem to reveal their professional personas on Twitter. Ninety percent of
doctoral students funded by the Canadian Social Sciences and Humanities Research Council (SSHRC)
identified as academics on Twitter (Work et al., 2015), 87% of surveyed US university professors
claimed to mention both their professional title and place of work in their Twitter profiles (Bowman,
2015b) and 78% of Twitter users who self-identified as a physicians used their full names (Chretien et
al., 2011). This willingness to reveal their scholarly identities on Twitter suggests that scholars make
use of the microblogging platform in a professional context at least to some extent.
Ke, Ahn, and Sugimoto (2017) took advantage of crowdsourced Twitter lists to identify scholars.
Based on a method introduced by Sharma, Ghosh, Benevenuto, Ganguly, and Gummadi (2012), they
identified scientists on Twitter with an approach based on membership in scientific Twitter user lists.
Other studies have estimated Twitter activity by scholars by analyzing users who engage with scholarly
content on Twitter. Hadgu and Jäschke (2014) applied machine learning to automatically identify
scholars on Twitter based on a training set of users whose tweets contained a computer science
conference hashtag, while others selected users who have tweeted scientific papers (Alperin, 2015b;
Haustein, Bowman, & Costas, 2015a; Haustein & Costas, 2015b; Holmberg & Thelwall, 2014; Tsou,
Bowman, Ghazinejad, & Sugimoto, 2015). Since the latter type of studies focuses on categorizing who
is tweeting about scholarly contents rather than estimating Twitter uptake in academia, these studies
are discussed in more detail below.
1.2.1.2 Scientific conferences Twitter does particularly well in fostering communication among people participating in shared
experiences (Rogers, 2014), which may be why tweeting at scholarly conferences has been one of the
earliest and most popular uses of Twitter in academia. Almost every scientific conference today has a
specific hashtag to connect attendees and those interested but not able to attend in person, thus
expanding the conference audience to include remote participants (Bonetta, 2009; Sopan, Rey, Butler,
& Shneiderman, 2012; Weller, Dröge, & Puschmann, 2011). Apart from increasing the visibility of
presentations, tweeting at scientific conferences has introduced another level of communication,
creating backchannel discussions online among participants complementing presentations and
discussions which take place at the meeting. Conference tweets usually directly refer to presentations
and discussions during sessions and sometimes summarize key take-away points (Chaudhry, Glode,
Gillman, & Miller, 2012; McKendrick, Cumming, & Lee, 2012; Mishori, Levy, & Donvan, 2014).
Other motivations to tweet at a scientific conferences were to share information and learn about
discussions in parallel sessions, networking with others and feeling a sense of connectedness, as well
as note-taking (McKendrick et al., 2012). A significant number of tweets associated with two medical
conference were uninformative or promotional (Cochran, Kao, Gusani, Suliburk, & Nwomeh, 2014;
Desai et al., 2012).
Handbook of Quantitative Science and Technology Research
6
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Due to the ease of collecting tweets with a particular hashtag, as well as Twitter’s relative popularity
in the context of scientific conferences, there are countless studies analyzing scholarly Twitter use
based on tweets with conference hashtags (Chaudhry et al., 2012; Cochran et al., 2014; Ferguson et
al., 2014; Hawkins, Duszak, & Rawson, 2014; Hawkins et al., 2014; Jalali & Wood, 2013; Letierce,
Passant, Breslin, & Decker, 2010; McKendrick et al., 2012; Mishori et al., 2014; Reinhardt, Ebner,
Beham, & Costa, 2009; Sopan et al., 2012; Weller et al., 2011).
Similar to the overall uptake among scholars, Twitter activity at conferences differs among disciplines
as well as individual conferences and increased over the years. Overall, only a small share of
conference participants contributed to discussions on Twitter: Less than 2% of attendees of the
American Society of Nephrology's 2011 conference (Desai et al., 2012) and less than 3% of participants
of the 2012 Winter Scientific Meeting of the Association of Anesthetists of Great Britain and Ireland
tweeted (McKendrick et al., 2012). Another medical conference in 2013 reported higher Twitter
engagement, as 13% of conference attendees tweeted using the conference hashtag (Mishori et al.,
2014). Longitudinal studies also observed an increase in Twitter activity at conferences over the years
(Chaudhry et al., 2012; Hawkins et al., 2014; Mishori et al., 2014). For example, 2% of conference
participants tweeted at the 2010 Annual Meeting of the American Society of Clinical Oncology, while
5% contributed to conference tweets in 2011. Similarly, the number of tweets nearly doubled from
4,456 tweets in the first to 8,188 in the following year (Chaudhry et al., 2012). A similar increase was
observed for the 2011 and 2012 annual meetings of the Radiological Society of North America
(Hawkins et al., 2014). Conference-related discussions on Twitter are not restricted to in-person
attendees. In fact, at some conferences the majority of Twitter users only participate remotely (Sopan
et al., 2012).
Just as with other social media and information in general, tweeting activity is usually heavily skewed
with a few users contributing the majority of tweets at conferences (Chaudhry et al., 2012; Cochran et
al., 2014; Mishori et al., 2014). Tweeting about a conference has been shown to lead to an increase in
the number of followers regardless of attending in-person or remotely. Follower counts grew
particularly for speakers and in-person attendees, while the number of followers grew least for remote
participants (Sopan et al., 2012). Most organizers of scientific conferences embrace the potential of
increasing visibility and outreach and thus encourage tweeting through a conference-specific hashtag.
Some also specifically display conference-related tweets in real time and thus make tweeting activity
visual to participants who are not on Twitter (Ferguson et al., 2014; Jalali & Wood, 2013; Sopan et al.,
2012; Weller et al., 2011).
1.2.1.3 Journals and publishers Twitter’s technological features afford direct connections and two-way conversations between users
changing what was traditionally known as a unidirectional sender-audience relationship. Opposed to
traditional publishing and mass media, Twitter has given rise to personal publics of audiences
(Schmidt, 2014). This direct link between the sender and receiver has changed the relationship with
audiences; for example, musicians use Twitter to market their own brand and respond to @replies from
fans to seek out in-person interactions (Baym, 2014). TV audiences turn Twitter into a virtual lounge
room when they connect with other users discussing TV events in real time (Harrington, 2014).
Similarly, discussions of scientific publications can now happen publically, when readers share their
opinions on Twitter. A specific use case are Twitter journal clubs, an adaption of small-group in-person
journal clubs that are common particular in the medical sciences (Leung, Siassakos, & Khan, 2015;
Mehta & Flickinger, 2014; Thangasamy et al., 2014; Topf & Hiremath, 2015; Whitburn, Walshe, &
Sleeman, 2015). Twitter journal clubs are used to discuss and review recent publications and educate
researchers and practitioners; in the medical sciences they also have the advantage over their offline
predecessors to directly involve patients (Mehta & Flickinger, 2014). Often these journal clubs are
initiated or at least supported by journals to promote their publications. A journal club initiated by a
Handbook of Quantitative Science and Technology Research
7
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
gynecology journal showed that discussing papers and making them freely available has boosted their
Altmetric scores (Leung et al., 2015). Twitter journal clubs also motivated authors of discussed papers
to create Twitter accounts (Thangasamy et al., 2014).
With journals and authors on Twitter, readers can get in touch directly and involve them in discussions
using @mentions, tearing cracks in the wall of traditional gatekeeping, as “Twitter makes it possible
to directly connect journal readers at various stages of training with authors and editors” (Mehta &
Flickinger, 2014, p. 1317). Many journals and publishers have started to use Twitter as a marketing
instrument to increase online visibility and promote published contents. These accounts can be used to
create a personalized audience relationship and to foster interaction among readers. Similar to the mix
of professional and personal interactions by academics on Twitter, the lines between scholarly
communication and marketing campaign are blurred for accounts maintained by journals and
publishers. Almost half of the 25 general medicine journals with the highest impact factor in 2010 had
a Twitter presence (Kamel Boulos & Anderson, 2012), while Twitter uptake was lower for other sets
of journals: 24% of 33 urology journals (Nason et al., 2015), 2 of the top 10 ophthalmology journals
(Micieli & Micieli, 2012), 16% of 100 Web of Science (WoS) journals (Kortelainen & Katvala, 2012)
and 14% of 102 journals specialized in dermatology (Amir et al., 2014) maintained an account. As
most of these studies focused on the top journals according to the journal impact factor, Twitter uptake
might be biased towards high-impact journals and slightly lower when including others. The variation
suggests similar differences between disciplines as observed for Twitter uptake by individual scholars
and conferences.
While most journal accounts are used to share articles and news (Zedda & Barbaro, 2015) and often
tweet the article title (Friedrich, Bowman, Stock, & Haustein, 2015; Thelwall, Tsou, Weingart,
Holmberg, & Haustein, 2013), some journals have incorporated tweeting into the formal
communication process. In addition to regular abstracts, they ask authors to write so-called tweetable
abstracts that meet the 140-character restrictions, which are used to attract readers on Twitter (Darling
et al., 2013). Twitter even interfered with the journal’s role in scholarly communication, when a
genomics paper was criticized and corrected results posted in a tweet, leading to a conflict with the
authors of the criticized paper (Woolston, 2015).
Even if a journal is not represented by a proper Twitter account, it is likely that its publisher is. Zedda
and Barbaro (2015) found that Twitter adoption was particularly common among 76 publishers in
science, technology and medicine; 89% had official Twitter accounts, exceeding the presence on any
other social media platform, and 74% had embedded tweet buttons that allowed readers to directly
share publications on Twitter. Promotion of publications by publishers seem to be welcomed by
authors, as a survey by Nature Publishing Group revealed that almost one fifth of authors would
consider it a very valuable service if publishers promoted papers using marketing and social media
(Nature Publishing Group, 2015).
As shown in the analysis of Twitter accounts diffusing scientific articles below, accounts maintained
by journals and publishing houses are responsible for a significant amount of tweets mentioning
scientific articles. Once Twitter metrics are being used to evaluate journal impact, these types of self-
tweets might be considered as a type of gaming in a manner similar to journal self-citations and citation
cartels to boost the impact factor (Seglen, 1997; Van Noorden, 2013). With publishers invested in the
success of their journals, tweet cartels and tweet stacking in analogy to their citation equivalents are
easily conceivable and even easier to implement. While the WoS excludes journals from the Journal
Citation Reports, which have been caught increasing their impact factors artificially, companies like
Altmetric.com and Plum Analytics do not (yet) intervene in such self-promotional activity.
Handbook of Quantitative Science and Technology Research
8
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
1.2.1.4 Universities and academic libraries
Scholarly institutions are affected by Twitter’s impact on academia on two levels: they exploit the
microblogging platform to increase their visibility (and that of their members) and provide guidelines
and recommendations for their members to navigate the new communication space. The Association
of American University Professors updated their report on Academic Freedom and Electronic
Communication (Association of American University Professors, 2013) in reaction to a university
rescinding a tenure-track job offer to an English scholar who had made an anti-Semitic comment on
Twitter (Herman, 2014). The updated report emphasized that professors enjoy academic freedom even
when they comment on social media and particularly addressed the blurring of boundaries between
private and professional opinions on social media. It stressed how, in this new context, comments are
particularly prone to be misunderstood and misinterpreted, as they are often taken out of context:
Electronic communications can be altered, or presented selectively, such that they are
decontextualized and take on implicit meanings different from their author’s original intent. With
the advent of social media such concerns about the widespread circulation and compromised
integrity of communications that in print might have been essentially private have only
multiplied further. (Association of American University Professors, 2013, p. 42)
The report further recommends that universities and other academic institutions, along with their staff,
develop policies that address the use of social media. In general, academic institutions lack specific
social media guidelines or address social media in policies. Although more and more institutions adopt
specific policies (Pasquini & Evangelopoulos, 2015), only half of US doctorate-granting universities
had a social media policy, while rates were even lower for other universities and colleges. At the same
time, Twitter was specifically mentioned in more than 80% of policies (Pomerantz, Hank, & Sugimoto,
2015).
The majority of university Twitter accounts apply a so-called megaphone model of communication,
where news and information concerning the institution are broadcasted following a traditional
communication model (Kimmons, Veletsianos, & Woodward, 2017; Linvill, McGee, & Hicks, 2012).
Universities use Twitter for public relations, dissemination of news and events, as well as recruitment
(Kimmons et al., 2017). Ninety-four percent of 474 US university admission officers reported that their
institution had a Twitter account (Barnes & Lescault, 2013) and 96% of the websites of 100 US
colleges linked to Twitter (Greenwood, 2012). On the departmental level, it was less common to be
represented with an organizational account, as only 8% of 183 US radiology departments had a Twitter
presence (Prabhu & Rosenkrantz, 2015). Twitter was also commonly used for faculty development at
medical schools (Cahn, Benjamin, & Shanahan, 2013). Analyzing the Twitter activity of 29 Israeli
universities and colleges, Forkosh-Baruch and Hershkovitz (2012) found significant differences
between both types of institutions. Colleges were more likely than universities to post social tweets.
While almost half of the tweets by universities focused on research conducted elsewhere, colleges
focused more on reporting the work by its own researchers.
Twitter was also frequently used by academic libraries as a marketing instrument, to communicate
with patrons, to announce new resources and promote services (Boateng & Quan Liu, 2014; Hussain,
2015; Shulman, Yep, & Tomé, 2015). On par with Facebook at a 63% adoption rate, Twitter was the
most commonly used social media platform among 38 surveyed academic libraries from different
countries (Chu & Du, 2013), while all of the 100 US university libraries analyzed by Boateng and
Quan Liu (2014) maintained a Twitter presences. In Canadian academic libraries Twitter adoption was
lower at 47% (Verishagen & Hank, 2014).
Handbook of Quantitative Science and Technology Research
9
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
1.2.2 Twitter use
Apart from Twitter uptake, scholarly Twitter metrics are further influenced by how Twitter is used.
Academics use social media to share information, for impression management and to increase their
visibility online, to network and establish a presence across platforms, to request and offer help, expand
learning opportunities, or simply to be social (Veletsianos, 2012). Twitter specifically was used mostly
to tweet work-related content, discover peers working on similar research, follow research-related
discussions and get recommendations for papers (Van Noorden, 2014).
One central motivation for scholars to tweet is to communicate and explain their work to lay people.
As many science communicators are active on Twitter, they help to bridge the gap between the
scholarly community and the general public. Science communicators were the largest user group of
518 Twitter users mentioned in tweets by 32 astrophysicists (Holmberg et al., 2014). An evolutionary
biology professor valued Twitter to communicate his work to the general public:
Twitter and regular blogging are more effective than anything else I do to publicize a paper,
which was really surprising to me […]. If you do it right, Twitter is an effective way of telling
people about your work. (Bonetta, 2009, p. 453)
However, most researchers still preferred traditional media over Twitter to promote their research
(Wilkinson & Weitkamp, 2013).
Even when identifying professionally on Twitter, a large share of tweets by scholars are not related to
their work or academia in general (Bowman, 2015b; Haustein, Bowman, Holmberg, Peters, &
Larivière, 2014; Mou, 2014; Pscheida et al., 2013; Van Noorden, 2014). The Nature survey found that
21% of scientists who used Twitter regularly did not use it professionally and 28% said that they never
posted content about their work (Van Noorden, 2014). Bowman (2015a, 2015b) reported that, while
29% of American university professors used Twitter strictly in a professional way and 42% used it for
both for personal and professional reasons, the vast majority of tweets were coded as personal (78%)
rather than professional (19%). Again, large variations can be observed between disciplines as well as
individual Twitter users (Chretien et al., 2011; Holmberg & Thelwall, 2014; Loeb et al., 2014; Mou,
2014; Priem, Costello, et al., 2012; Work et al., 2015). Examining more than half a million tweets from
447 researchers, Holmberg and Thelwall (2014) found that less than 4% of tweets were classified as
scholarly communication and results varied between disciplines ranging from less than 1% for
sociology up to 34% for biochemistry. A study on emergency physicians’ tweeting behavior found that
49% of their tweets were related to health or medical issues, 21% were personal, 12% self-promotional
and 3% considered unprofessional as they contained profanity, were discriminatory or violated patient
privacy (Chretien et al., 2011). In a sample of tweets by funded doctoral students in the social sciences
and the humanities in Canada, 4% of tweets were related to their thesis, 21% to the discipline and 5%
to academia in general, while 70% of tweets were coded as non-academic (Work et al., 2015). Personal
use also prevailed among 382 urologists (Loeb et al., 2014).
These findings highlight that even when scholars identify professionally on Twitter and use the
platform for scholarly purposes, many tweets will be irrelevant to scholarly communication and should
thus be excluded from a scholarly indicator perspective:
The lack of a dividing line between scientists and non-scientists, as well as the great variety of
topics that even scientists tweet about mean that Twitter is not comparable to the orderly world
of science publishing, where every piece of information is assumed to be relevant. Instead, a
typical user’s timeline is likely to be populated both by scholarly content and personal remarks,
more or less side by side. (Puschmann, 2014, p. 98)
Handbook of Quantitative Science and Technology Research
10
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Due to their brevity and the fact that when analyzing tweets they are often taken out of context,
categorizing tweet content is as difficult as it is to classify Twitter users (Bowman, 2015b).
Distinguishing between scientific and non-scientific tweets is especially challenging (Holmberg &
Thelwall, 2014).
Large variations can also be found between individual tweeters in terms of how often they tweet. A
group of astrophysicists analyzed by Haustein et al. (2014) tweeted, on average, between 0 and 58
times per day. Tweets to scientific papers have been shown to peak shortly after their publication and
decay rapidly within just a few days. For example, 80% of arXiv submissions received the largest
number of tweets the day after they were published (Shuai, Pepe, & Bollen, 2012). Similarly tweeted
half-life was 0 days for papers published in the Journal of Medical Internet Research (Eysenbach,
2011) and 39% of a sample of tweets linking to a scholarly document referred to those published
within one week before (Priem & Costello, 2010). Determining the delay between publication and first
tweet as well as half-lives on Twitter is, however, challenging due to the ambiguity of publication dates
(Haustein, Bowman, & Costas, 2015b).
Since not all Twitter use culminates in a tweet, a large share of activity remains invisible and thus
unmeasurable. In fact, passive use prevailed among UK doctoral students on Twitter (Carpenter et al.,
2012) and seems to be common for scholarly use of social media in general. While most academics
access and view information, only a minority actively contributes by creating content on social media
(Procter et al., 2010; Tenopir et al., 2013); less than 2% of 1,078 UK researchers surveyed actively
contributed daily (Tenopir et al., 2013).
1.2.2.1 Tweeting links
The most frequent use of Twitter among researchers in higher education was to share information,
resources or media (Veletsianos, 2012). A survey among US university professors revealed that
embedding URLs was the most commonly used Twitter affordance. Half of the survey participants
claimed to tweet links either sometimes, mostly or always (Bowman, 2015b). Links are a common
way to send more information than 140 characters would fit. Addressing the length limitation, a scholar
explained:
It is a double-edged sword. The majority of my tweets are pointers to other resources, so there is
a headline—an enticement in other words—and a link to the resource. You don’t need more than
140 characters for that. (Bonetta, 2009, p. 453)
Weller and Puschmann (2011) refer to links in tweets as “external citations” (Weller & Puschmann,
2011, p. 2). Studies about scholars on Twitter show that they make frequent use of tweeting URLs, as
the share of tweets with links exceeds that observed for general Twitter users (boyd et al., 2010;
Harrington, 2014). About one third of 68,232 tweets sent by 37 astrophysicists (Haustein, Bowman,
Holmberg, et al., 2014) and 38% of 22,258 tweets posted by Canadian social sciences and humanities
doctoral students contained links (Work et al., 2015). Tweeting links was even more popular among
scholars studied by Weller and Puschmann (2011) and emergency physicians analyzed by Chretien et
al. (2011), as respectively 55% and 58% of their tweets contained URLs. Links were much more
common when a sample of 445 US professors tweeted professionally: 69% of professional tweets
contained URLs, while only 15% of personal tweets did (Bowman, 2015b). Tweets with the
#www2010 and #mla09 conference hashtags linked to a website in 40% and 27% of the cases,
repeating each unique URL less than three times (Weller et al., 2011).
Priem and Costello (2010) found that 6% of a sample of 2,322 of tweets by academics containing a
URL mentioned a scholarly publication, 52% of which were first-order and 48% second-order links
(i.e., via another website) to the document. Similarly, Holmberg and Thelwall (2014) found that
scholarly tweets frequently contained a link to scholarly publications via a blog post about the paper.
Handbook of Quantitative Science and Technology Research
11
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
First-order links were significantly more likely to refer to open access articles (Priem & Costello,
2010). Tweets containing the #iswc2009 conference hashtag linked to applications (e.g., online
services or research projects; 31%), the conference website (21%), blog posts (12%), slideshows (12%)
and publications (9%) (Letierce et al., 2010). Blogs were the most common linked resources in
conference tweets analyzed by Weller and colleagues (2011), while news websites were a frequent
link destination of tweets sent by Canadian doctoral students, even when discussing scholarly topics
(Work et al., 2015).
When linking to scholarly papers, tweets often contained the paper title and rarely expressed any
recommendation or sentiment (Friedrich, Bowman, Stock, et al., 2015; Thelwall et al., 2013). The
great majority of tweeted articles were published very recently (Eysenbach, 2011; Holmberg &
Thelwall, 2014; Priem & Costello, 2010). According to surveys asking for motivations to use social
media, finding relevant publications and staying up-to-date with the literature was found to be a
frequent, yet passive, use of Twitter (Van Noorden, 2014). A Columbia university professor in biology
and chemistry describes how they used Twitter to be alerted about the literature:
Sometimes four or five people I follow will mention a paper that I did not come across and I will
look it up. I think I am much more up to date on science literature since I started following
Twitter. (Bonetta, 2009, p. 452)
A study by Tenopir, Volentine, and King (2013) found that academics on Twitter read more scholarly
publications, which seems to confirm the use of Twitter as a publication alert service. At the same
time, as the analyses by Priem and Costello (2010) and Letierce et al. (2010) show, the share of tweets
linking to academic papers is low, suggesting a rather passive use: scholars follow links to tweeted
articles but do only infrequently distribute them themselves. This suggests that a significant part of
Twitter use cannot be captured by scholarly Twitter metrics.
1.2.2.2 Retweets
Retweets represent a specific form of diffusing information, as users forward messages sent by others.
As such, they do not represent an original contribution by the retweeting user. Since retweets directly
quote another users text, they can be seen as “internal citations” (Weller & Puschmann, 2011, p. 3) on
Twitter. An analysis of retweets demonstrates how information circulates within a specific user
community (Paßmann, Boeschoten, & Schäfer, 2014). A common disclaimer that retweets do not equal
endorsements adapted from early Twitter use by journalists, emphasizes that tweets are forwarded to
increase information diffusion. Once a frequent part of Twitter bios, the disclaimer has now been
established as common sense and is no longer needed (Warzel, 2014).
As sharing information is one of the main motivations for scholarly Twitter use, retweeting is likely to
be common among tweeting academics. Conference participants interviewed by Letierce and
colleagues retweeted “tweets that are close to their interest or tweets that speak about their own work
or research project” (Letierce et al., 2010, p. 7). Studies showed that retweeting is less common than
other affordances used by scholars on Twitter but exceed expectations of a random sample of tweets
in 2009, in which as few as 3% were retweets (boyd et al., 2010). Between 15% and 20% of tweets
sent at scientific conferences were retweets (Letierce et al., 2010); similarly, 15% of 68,232 tweets by
a group of 37 astrophysicists were retweets (Haustein, Bowman, Holmberg, et al., 2014). Retweeting
was more common among 28 academics analyzed by Priem and Costello (2010), as 40% of tweets
were retweets. Similarly, 37% of 43,176 tweets by Canadian doctoral students were retweets, while
10% of their tweets were retweeted (Work et al., 2015). At the same time, the share of retweets at a
radiology conference was 60% (Hawkins et al., 2014). Being asked about Twitter affordance use, 14%
of US professors said that they mostly or always and 34% that they sometimes retweeted (Bowman,
2015b). These professors were more than twice as likely to retweet when their tweets were classified
Handbook of Quantitative Science and Technology Research
12
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
as professional rather than personal (Bowman, 2015b). These differences again demonstrate that
tweeting behavior differs depending on who is tweeting and it what context.
The majority of retweets sent by scholars from ten disciplines contained links, while conversational
tweets (i.e., @mentions) were less likely to contain links.
This clearly shows that researchers […] frequently share web content and forward information
and content they have received from people they follow on Twitter, while links are not that often
shared in conversations. (Holmberg & Thelwall, 2014, p. 1035)
Links to papers were significantly less likely to be retweeted: while 19% of tweets with links to
scholarly publications were retweeted, the retweet rate was twice as high in the overall sample of
tweets analyzed by Priem and Costello (2010). This is in contrast to a random sample of more than
200,000 tweets, over half of which contained a URL (boyd et al., 2010). A random sample of 270
tweets linking to scientific journal articles found that many were modified retweets of tweets
originating from the journal’s own Twitter account (Thelwall et al., 2013).
From the perspective of scholarly metrics, a distinction should arguably be made between tweets and
retweets, as the latter reflects a rather passive act of information sharing (Holmberg, 2014). Although
with each retweet the visibility of the tweet and the information it contains (e.g., the link to a
publication) increases, retweets represent diffusion of information rather than impact. As retweeting
requires even less effortas little as one click since the implementation of the retweet buttonthan
composing an original tweet, Twitter metrics should distinguish between tweets and retweets to reflect
these different levels of user engagement (Haustein, Bowman, & Costas, 2016).
1.2.2.3 Followers, @mentions and @replies
Scholars on Twitter actively seek new connections and connect others (Veletsianos, 2012). Twitter is
built in a way that information is spread via user networks. This means that users receive updates from
those they follow and diffuse their messages to those they are followed by, creating a personal public
(Schmidt, 2014). Selecting who to follow and who one is followed by are thus essential to
communicating on Twitter. Twitter users build a reputation based on both their number of followers
and followees. A parallel can be drawn to authors’ citation identity and citation image based on which
authors one cites and is cited by (White, 2000, 2001).
The purpose of expanding one’s social network and finding peers was apparent in a study of 632
emergency physicians on Twitter: those who included work-related information in their Twitter bios
had more followers and the most influential users in the network were connected to at least 50 other
emergency physicians (Lulic & Kovic, 2013). Similarly, conference participants on Twitter saw a
significant increase in their number of followers, particularly when they were also presenting (Sopan
et al., 2012). The average number of followers of 260 physicians analyzed by Chretien et al. (2011)
was 17,217 with a median of 1,426, indicating the typically skewed distribution of followers among
Twitter users.
In addition to broadcasting one’s message via retweets on a meso-level of communication, Twitter
users can also directly address users with @mentions or @replies on an interpersonal level. Both
replies and mentions thus represent a particular type of tweet that focuses on conversation rather than
broadcasting (Honeycutt & Herring, 2009). Like most Twitter affordances, these types of tweets were
also developed by the user base before being implemented by Twitter. While @replies happen in
response to a tweet and are only visible on the timeline of the tweeter who sent the original tweet,
@mentions refer to tweets that contains another user’s Twitter handle, which triggers a notification to
inform them about being mentioned (Bruns & Moe, 2014). About one third of tweets in 2009 included
another user name (boyd et al., 2010). A random sample of tweets without an @mention were mostly
Handbook of Quantitative Science and Technology Research
13
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
about the tweeting user’s experience, while those with an @mention were more likely about the
addressee. In fact, more than 90% of @mentions functioned to address another user, while 5% worked
as a reference (Honeycutt & Herring, 2009).
The great majority of tweets sent by social sciences and humanities doctoral students referred to other
Twitter users, as 72% of the 43,176 tweets sent contained other user names (Work et al., 2015).
Conversational tweets were also popular among a group of tweeting astrophysicists. Of the 68,232
tweets 46% were @replies or @mentions (61% including RTs), making it the most frequently used
Twitter affordance. Conversational tweets were particularly common among those who tweeted
regularly or frequently (Haustein, Bowman, Holmberg, et al., 2014). Most of these mentions referred
to science communicators (24%), other astrophysicists (22%) or organizations (13%) on Twitter
(Holmberg et al., 2014).
Conversational tweets hardly contained links (Holmberg & Thelwall, 2014), and among tweets linking
to publications, only 8% were @replies (Priem & Costello, 2010). Opposed to hashtag use, retweeting
and embedding URLs, @mentions were the only affordance that were less likely to occur in professors’
professional tweets (56%) than those identified as personal (67%), which suggests that when professors
discuss their work, they are less likely to address or reference other users directly than when they tweet
about private matters. However, mentioning other users was still more common in their professional
tweets than retweeting and using hashtags (Bowman, 2015b).
1.2.2.4 Hashtags
Similar to retweets and @mentions, hashtags are a user-driven Twitter affordance. Hashtags are
keywords following the #-sign, which facilitate connections between users interested in the same
topics. Conversations revolving around hashtags represent the macro layer of Twitter communication
(Bruns & Moe, 2014). Holmberg and colleagues suggest that “hashtags may resemble the traditional
function of metadata by enhancing the description and retrievability of documents” (Holmberg et al.,
2014, p. 3).
Hashtag use seem to be less common than that of other Twitter affordances among academics. Sixty-
one percent of surveyed American university professors declared that they rarely or never used a
hashtag (Bowman, 2015b). This might be because scholars are either less familiar with this Twitter-
specific affordance or they do not wish to expand conversations beyond their personal publics defined
by their follower networks. Although actual hashtag use was low by the US professors analyzed by
Bowman (2015b), they were more likely to use hashtags in their tweets identified as professional
(28%) than those coded as personal (17%).
An early large-scale study found that among a random sample of 720,000 tweets, only 5% contained
a hashtag (boyd et al., 2010). Almost one quarter of tweets by astrophysicists contained a hashtag
(Haustein, Bowman, Holmberg, et al., 2014), but hashtag use varied significantly among different
clusters of the follower network (Holmberg et al., 2014). The same share of hashtags (25%) was found
for tweets of SSHRC funded doctoral students; on average each hashtag was mentioned 2.8 times
(Work et al., 2015). It is problematic to infer hashtag use from most other studies on scholarly tweets,
as data collection itself is often based on a specific hashtag, such as a conference hashtag (e.g., Letierce
et al., 2010; Weller et al., 2011).
1.2.3 Reluctance against and negative consequences of using Twitter
When using tweets as the basis to measure scholarly impact of any sorts, it is essential to consider who
is not on Twitter and why academics might be reluctant to join the microblogging platform. Twitter is
a platform where “content is not king” (V. Miller, 2008, p. 395) and has been perceived as “shallow
media, in the sense that it favors the present, popular and the ephemeral” (Rogers, 2014, p. xiv). Early
Twitter studies which identified the majority of tweets to be “pointless babble” (Kelly, 2009, p. 5) or
Handbook of Quantitative Science and Technology Research
14
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
“daily chatter” (Java, Song, Finin, & Tseng, 2007, p. 62) casted doubt on the value of Twitter as a
meaningful communication medium (Rogers, 2014). This reduction to banal content has led many in
academia to consider tweeting a waste of time and therefore rejecting Twitter as a means of scholarly
communication (Gerber, 2012). Particularly the 140-character limit of tweets has many scholars doubt
Twitter’s usefulness for research. This may be why Twitter is one of the best known and at the same
time least used social media platforms in the scholarly community (Carpenter et al., 2012; Gerber,
2012; Gu & Widén-Wulff, 2011; Pscheida et al., 2013; Van Noorden, 2014).
The adoption of new technologies is often met with functional and psychological barriers. Lack of time
and skills as well as a negative perception of platforms have been identified as barriers to use social
media in academia (Donelan, 2016). Reluctance often stems from the notion that tweeting wastes
precious time and introduces challenges that come with the blurred boundaries of professional and
personal communication. This mix of professional and personal identities on Twitter specifically has
been revealed by many studies and has been identified as a potential reason not to use Twitter in
academia. Even though in the general public, Twitter uptake is higher among young adults, it is often
early career researchers who are more reluctant to tweet about their work (Bulger et al., 2011;
Coverdale, 2011). Young academics feel the highest pressure to publish in high-impact journals and
limit their time spent on social media, and feel more vulnerable when publicly exposing their ideas,
particularly to uncertain audiences (Coverdale, 2011; Harley, Acord, Earl-Novell, Lawrence, & King,
2010; Mccrea, 2011; Veletsianos, 2013). An academic interviewed about the future of scholarly
communication expressed their concern regarding the use of Facebook and Twitter for work purposes:
There’s this research group in my area and, for some reason, they’re really into Facebook. So
they want to do a lot of discussions on Facebook and that type of thing. But I really just don’t
have time. It’s like Twittering. I just can’t…There needs to be a little bit of space where I can
actually think about something. And I think for some people, they’re just wired in such a way
that they like that constancy, and they are also able to actually say something intelligent quickly.
And I’m not like that. I have to be a little bit more deliberate and think about things a little bit
more. And so I can’t Twitter…I need some time to reason… (Harley et al., 2010, p. 97)
However, early-career researchers were often more likely to find social media useful in the context of
scholarly communication and collaboration (Grande et al., 2014; Gruzd, Staves, & Wilk, 2012; Tenopir
et al., 2013). Bowman (2015b) identified a u-shaped relationship between Twitter use and academic
experience: US professors seven to nine years into their academic careers were more likely to use
Twitter compared to those with fewer and more years of experience. A humanities scholar addressed
the issue of blurred boundaries on Twitter:
I think it can be distracting, especially to grad students, when they’re trying to navigate, when
they’re needing to learn, adopt, and use these new technologies, but at the same time learn to
discriminate among technologies that are more for social things but are being used in the name
of research. The lines are too blurry. (Bulger et al., 2011, p. 59)
The tension that arises between scholars who take to Twitter and those who are reluctant to discuss
scholarly matters on social media is demonstrated by an incident where a genomics paper published in
PNAS was criticized on Twitter for flaws in study design and analysis casting doubt on its conclusions.
The tweet sent by a genetics researcher at the University of Chicago included charts and tables from a
re-analysis of the PNAS paper’s data, which he published in the open access and open-peer review
journal F1000Research. The tweet and re-analysis provoked many responses on Twitter and in the
comment section of the F1000Research paper, some of which demanded the retraction of the PNAS
paper (Woolston, 2015), while the PNAS authors accused the critic of violating the norms of science
by taking to Twitter.
Handbook of Quantitative Science and Technology Research
15
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
The clashing of personal and professional and the fuzzy boundaries between the two has also lead to
severe negative consequences for scholars. Tweets by faculty have caused outrages among students,
other faculty members, university administration and the public at large. Identifying professionally on
Twitter has affected the academic careers of some scholars and, in a certain case, controversial tweets
have provoked death threads (Rothschild & Unglesbee, 2013). Tweeting had serious effects on the
career of a tenured University of New Mexico psychology professor, who had fat-shamed students on
Twitter: “Dear obese PhD applicants: if you didn’t have the willpower to stop eating carbs, you
won’t have the willpower to do a dissertation #truth.” The professor was asked to apologize and had
to undergo sensitivity training, while his work was monitored and he was banned from working on the
graduate students admission committee for the rest of his career (Bennett-Smith, 2013; Ingeno, 2013;
Pomerantz et al., 2015).
In another incident, an English professor lost his tenure-track position offer from the University of
Illinois at Urbana-Champaign due to a tweet that was interpreted as anti-Semitic (Herman, 2014). In
response to withdrawal of the job offer by the University of Illinois, the American Association
University Professors updated their report on academic freedom to reinstate that academic freedom
applies to comments made on social media (Association of American University Professors, 2013).
1.3 Scholarly output on Twitter As this chapter discusses scholarly Twitter metrics, it focuses on how scientific papers are diffused via
tweets. Although, as described above, only a small amount of scholars’ tweeting activity involves
linking to publications (Letierce et al., 2010; Priem & Costello, 2010), tweets to journal articles
represents one of―if not the―most popular altmetric. This might be due to the significance of
publications in peer-reviewed journals in scholarly communication as well as the ease at which tweets
that mention or link to document identifiers (e.g., the Digital Object Identifier DOI) can be retrieved.
Another reason that Twitter-based altmetrics―and, in fact, all altmetrics―gravitate towards journal
articles is that they aim to complement existing bibliometric measures, which reduce scholarly output
in a similar manner.
In the following, the altmetrics literature is reviewed to provide an overview of currently used scholarly
Twitter metrics for journal articles. As the majority of available studies only scratch the surface of
what can potentially be extracted from Twitter activity, the literature review is complemented by an
analysis of tweets collected by Altmetric.com. This analysis goes beyond tweet counts and correlations
with citations and aims to reflect the What, How, When, Where and Who of scholarly publications
shared on Twitter. This includes what types of documents are tweeted, how Twitter affordances such
as hashtags, retweets and @mentions, are used to share them, when and where articles are tweeted and
who is tweeting them. Although this chapter is not able to provide solutions to problems associated
with creating meaningful metrics from tweets, particular limitations and pitfalls are highlighted, while
demonstrating the potential of available data. Together with the findings on Twitter use described
above, the chapter tries to provide context to help the interpretation of different metrics. This chapter
thus aims at improving the understanding of what Twitter-based scholarly metrics can and cannot
reflect.
1.3.1 Data and indicators
The analysis of tweets that mention scholarly documents is based on Twitter data collected by
Altmetric.com until June 2016, which contains 24.3 million tweets mentioning 3.9 unique documents.
Altmetric started systematically collecting online mentions of scholarly publications in 2012 and is a
particularly valuable data source for tracking Twitter activity related to scholarly output, as it
continuously stores tweets that mention scholarly publications with a DOI. Through accessing tweets
through the Twitter API firehose, Altmetric circumvents the usual issues of Twitter data collection that
researchers are confronted with when using the freely available Twitter APIs. While the Twitter
Streaming API limits access to a random sample of 1% of tweets, the Representational State Transfer
Handbook of Quantitative Science and Technology Research
16
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
(REST) API is rate-limited and the Search API restricts access to only the most recent tweets relevant
to a particular query (Gaffney & Puschmann, 2014). As Altmetric started data collection in 2012,
Twitter activity is incomplete for documents published earlier.
Table 1 Scholarly Twitter metrics associated with scholarly documents
Type of
metric
Scholarly Twitter
metric
Description
Tweets
Twitter coverage Percentage of documents with at least one tweet
Number of tweets Sum of total number of tweets
Twitter density Mean number of tweets per document
Twitter intensity Mean number of tweets per tweeted document
Retweets
Share of retweets Percentage of tweets that were retweets
Retweet density /
intensity
Mean number of retweets per document / tweeted
document
Users
Number of users Unique number of users associated with a document
User density / intensity Mean number of users per document / tweeted
document
Mean number of
followers
Mean of the number of followers of users tweeting a
document
Hashtags
Hashtag coverage Percentage of documents with at least one hashtag
Number of hashtags Unique number of hashtags associated with a
document
Hashtag frequency Sum of total number of hashtag occurrences
Share of hashtags Percentage of tweets with at least one hashtag
Hashtag density /
intensity
Mean number of hashtags per document / tweeted
document
@mentions
@mention coverage Percentage of documents mentioning a user name
Number of mentioned
users
Unique number of users mentioned in tweets
associated with a document
@mention frequency Sum of total number of @mentions
@mention density /
intensity
Mean number of @mentions per document / tweeted
document
Time
Tweet span Number of days between first and last tweet
Tweet delay Number of days between publication of a document
and its first tweet
Twitter half-life Number of days until 50% of all tweets have appeared
Altmetric’s Twitter data is matched to bibliographic information from WoS using the DOI. This match
between document metadata and tweets affords the possibility to determine the amount of scholarly
output that does and does not get tweeted. The link to WoS data also provides access to cleaner and
extended metadata of tweeted documents, such as the publication year, journal, authors and their
affiliations, and a classification system of scientific disciplines. At the same time, the match of the two
databases also excludes tweets to publications not indexed in WoS and thus comes with the known
restriction and biases of WoS coverage. This is why the following analysis describes results for two
datasets, containing, respectively all 24.3 million tweets covered by Altmetric (dataset A), and 3.9
million tweets mentioning documents with a DOI, covered by WoS and published in 2015 (dataset B).
The number of unique documents in dataset A is based on Altmetric ID. As Altmetric.com’s metadata
is based on multiple sources, one publication might be treated as two documents, particularly if it has
Handbook of Quantitative Science and Technology Research
17
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
multiple versions, such as a journal article on the publisher’s website and a preprint on arXiv. Similar
duplications are possible but less likely in dataset B, which is based on unique identifiers and cleaner
metadata in WoS.
In the following, a set of descriptive indicators are used based on tweets to scholarly documents and
associated metadata. Table 1 provides an overview of each metric. As described above, the focus here
is on journal articles but the metrics can, nevertheless, be applied to any scholarly document or other
research object such as scholarly agents including “individual scholars, research groups, departments,
universities, funding organizations and others entities acting within the scholarly community”
(Haustein, Bowman, & Costas, 2016, p. 376). Similar to most bibliometrics, the metrics described in
Table 1 can be applied to any aggregated set of documents, such as all documents relevant to a certain
topic, published in the same journal, by the same author, institution, country or in a specific language.
1.3.2 What scholarly output is tweeted?
As shown above, only a small share of tweets sent by scholars actually link to scholarly output. Tweets
linking to blogs or other websites are often more frequent than those linking to scientific publications.
At the same time, the great majority of altmetric studies on Twitter focus on peer-reviewed scientific
journal articles with a DOI, which represent one of the main limitations of currently captured altmetrics
(Alperin, 2013; Haustein, 2016; Haustein, Sugimoto, & Larivière, 2015; Taylor, 2013).
In comparison to other common altmetrics, Twitter is the platform which exhibits the second largest
activity related to scientific papers, following the social reference management platform Mendeley
(Costas, Zahedi, & Wouters, 2015b; Haustein, Costas, et al., 2015). The disciplinary differences in
Twitter uptake described above are equally visible, and likely resulting in, large difference in Twitter
coverage between scientific disciplines. In the majority of studies, usually between 10% to 30% of
selected documents were mentioned on Twitter at least once (Alhoori & Furuta, 2014; Andersen &
Haustein, 2015; Costas et al., 2015a; Hammarfelt, 2014; Haustein, Costas, et al., 2015; Haustein,
Larivière, Thelwall, et al., 2014; Haustein, Peters, Sugimoto, et al., 2014; Priem, Piwowar, et al., 2012).
As the majority of tweets linking to scholarly papers occurs immediately after publication (Eysenbach,
2011; Shuai et al., 2012) and Twitter activity increased annually, Twitter coverage increases by year
of publication, with the most recent papers being more likely to be tweeted and older papers hardly
getting shared on Twitter (Haustein, Peters, Sugimoto, et al., 2014). For example, while more than half
of 2015 PLOS papers were tweeted at least once (Barthel et al., 2015), Twitter coverage was at 12%
for those published in 2012 (Priem, Piwowar, et al., 2012). Similarly, 13% of 2011(Costas et al.,
2015a), 16% of 2011-2013 (Robinson-García, Torres-Salinas, Zahedi, & Costas, 2014) and 22% of
2012 WoS documents (Haustein, Costas, et al., 2015) had received at least one tweet, while Twitter
coverage increased to 36% for WoS 2015 papers in dataset B (see Table 2 below).
Coverage varied between disciplines and journals, but also between databases and geographic regions.
Twitter is blocked in countries like Iran and China, which reflects on the visibility of their authors and
papers. For example, the share of tweeted papers was low for papers published by authors from Iran
(Maleki, 2014). Such geographical biases affect Twitter visibility and need to be taken into account
when comparing Twitter impact of documents, authors and institutions from different countries. For
example, as few as 6% of Brazilian documents published 2013 and indexed in SciELO (Alperin,
2015a) and 2% of a sample of Iranian papers covered by WoS had been tweeted (Maleki, 2014). On
the contrary, at 21% Twitter coverage of Swedish publications seems to be more in line with general
findings (Hammarfelt, 2014). As the sample of Iranian publication included documents published
between 1997 and 2012 and 98% of tweeted papers were published between 2010 and 2012, both
geographical and publication date biases influence Twitter coverage.
Particularly high coverage was found for arXiv submissions (Haustein, Bowman, Macaluso, Sugimoto,
& Larivière, 2014; Shuai et al., 2012). This high activity was, however, not caused by high Twitter
Handbook of Quantitative Science and Technology Research
18
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
uptake in the physics, mathematics and computer science communities, but created by Twitter bots
(Haustein, Bowman, Holmberg, et al., 2016). The extend of such automated rather than human activity
and its implications for the meaning of scholarly Twitter metrics are further discussed below.
While Twitter coverage describes the extent to which a set of documents gets diffused on Twitter,
Twitter density (i.e., the mean number of tweets per paper) reflects the average tweeting activity per
document. In general, each paper receives less than one tweet on average, with large variations between
disciplines (see below). As Twitter density is influenced by Twitter coverage and thus particularly low
when only a small share of papers gets tweeted, Twitter intensity reflects the average tweeting activity
for tweeted papers only, excluding non-tweeted papers (Haustein, Costas, et al., 2015). The number of
tweets per paper is usually heavily skewed, much more than citations. For example, 63% of tweeted
biomedical papers were only tweeted once (Haustein, Peters, Sugimoto, et al., 2014).
Figure 1 Number of tweets and users per tweeted document for datasets A and B.
A similar distribution can be observed for the two datasets described above (Figure 1). The 24.3 million
tweets in dataset A mentioned 3.9 documents (based on Altmetric ID), 43% of which were tweeted
once, 19% twice, 10% three times, while only 4% of documents received more than 25 tweets. The
document with the most Twitter activity was mentioned 35,135 times by 144 users, with one user
tweeting 34,797 times. This highlights that the number of distinct users per document might be a better
proxy of diffusion than the total number of tweets.
Tweets in dataset A were sent by 2.6 million users and link to 3.9 million documents, amounting to a
Twitter intensity of 6.2 tweets per tweeted paper. The number of users is based on unique Twitter
handles included as Author ID on Source in Altmetric’s Twitter data. Since the data does not include
Twitter’s unique user ID, users might be counted more than once if they changed their Twitter handle.
Very likely caused by the 140-character limitation, less than 1% of tweets link to more than one
document. As shown in Figure 1, the distribution of number of users per document is even more
skewed with 47% of all documents tweeted by a single user only. The distribution for WoS 2015 papers
(dataset B) is similar but slightly less skewed for documents (based on WoS identifier) mentioned in
one tweet and by one user only; 35% and 38% of documents were tweeted once or by one user,
respectively. The total number of 3.9 million tweets were sent by 601,290 users mentioning 548,841
documents, which corresponds to a Twitter coverage of 36%, intensity of 7.2 and a density of 2.6
tweets per document.
1.3.2.1 Disciplines and journals
Twitter activity varies among disciplines and even journals of the same field. As there is no
gatekeeping in Twitter, these variations might not be entirely due to actual impact of a particular
journal but can be heavily influenced by individuals and marketing strategies of publishing houses or
other stakeholders. Despite these variations, studies have shown that multidisciplinary and biomedical
journals as well as social science publications are particularly visible on Twitter, while the so-called
hard sciences are tweeted about less (Alhoori & Furuta, 2014; Andersen & Haustein, 2015; Barthel et
al., 2015; Costas et al., 2015b; Fenner, 2013; Hammarfelt, 2014; Haustein, Costas, et al., 2015;
Handbook of Quantitative Science and Technology Research
19
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Haustein, Larivière, Thelwall, Amyot, & Peters, 2014; Haustein, Peters, Sugimoto, et al., 2014; Knight,
2014; Priem, Piwowar, et al., 2012).
Previous findings are corroborated by the analysis of discipline and journal-based tweeting activity in
dataset B. As shown in Table 2, over one third of 2015 documents in WoS have been tweeted, which
represents a significant increase compared to papers published in the previous years. Twitter coverage
shows large variation between disciplines from 59% of publications in Biomedical Research, Health
and Psychology to less than 10% in Mathematics and Engineering & Technology. This upholds
previous findings that Twitter activity is particularly elevated around publications from the biomedical
and social sciences (Costas et al., 2015a; Haustein, Costas, et al., 2015).
Table 2 Dataset B: Twitter activity for WoS articles published in 2015.
Discipline
Papers 2015 Tweeted papers Tweets Users
N
DO
I
cov
erag
e
N
Tw
itte
r
cov
erag
e
N
Den
sity
Inte
nsi
ty
% R
Ts
N
Inte
nsi
ty
All disciplines 2,014,977 76% 548,841 36% 3,960,431 2.6 7.2 50% 601,290 6.6 N
atu
ral
Sci
ence
s &
Eng
inee
ring
Biology 124,402 73% 33,945 37% 226,575 2.5 6.7 54% 52,235 4.3
Biomedical Res. 226,011 84% 112,470 59% 1,025,061 5.4 9.1 50% 229,851 4.5
Chemistry 158,929 90% 32,829 23% 81,739 0.6 2.5 34% 13,860 5.9
Clinical Medicine 663,481 64% 223,641 52% 1,784,438 4.2 8.0 51% 288,226 6.2
Earth & Space 96,792 91% 25,616 29% 136,732 1.6 5.3 51% 42,641 3.2
Engr. & Tech. 253,020 88% 18,441 8% 50,577 0.2 2.7 33% 17,235 2.9
Mathematics 51,240 85% 2,890 7% 17,441 0.4 6.0 43% 7,826 2.2
Physics 128,766 93% 16,783 14% 55,295 0.5 3.3 30% 14,822 3.7
So
cial
Sci
ence
s
& H
um
anit
ies
Arts 18,995 19% 566 15% 1,962 0.5 3.5 39% 1,134 1.7
Health 51,535 74% 22,662 59% 191,530 5.0 8.5 52% 60,090 3.2
Humanities 76,998 39% 4,601 15% 19,150 0.6 4.2 52% 9,784 2.0
Prof. Fields 54,109 72% 14,760 38% 93,936 2.4 6.4 47% 41,449 2.3
Psychology 40,657 78% 18,735 59% 145,767 4.6 7.8 51% 50,974 2.9
Social Sciences 70,042 76% 20,902 39% 135,193 2.5 6.5 53% 51,288 2.6
It should be emphasized that Twitter activity in the Arts and Humanities cannot be generalized as DOIs
are not commonly used in these disciplines. While in general 76% of all documents had a DOI, as few
as 19% of all WoS-indexed journal articles in the Arts were linked to this unique identifier. DOI use
does not only differ between disciplines but also by country or language of publication, which is why
results may be biased in these regards as wells. Considering National Science Foundation (NSF)
specialties with more than half of its papers having a DOI, Twitter coverage was highest in Parasitology
(78%), Allergy (76%) and Tropical Medicine (70%) and lowest in Metals & Metallurgy (1%),
Miscellaneous Mathematics (2%), Mechanical Engineering (2%) and General Mathematics (3%).
With more than 10 tweets per document, Twitter density was highest in General & Internal Medicine
(13.5) and Miscellaneous Clinical Medicine (12.3).
At a Twitter coverage of 100%, 198 of 9,340 tweeted journals had all of their documents diffused on
Twitter, which strongly suggests a systematic and automated diffusion, possibly by a dedicated account
maintained by the journal or publisher (see below). Among journals with at least 100 papers with a
DOI in 2015, JAMA, and Biotechnology Advances had the highest Twitter density at 115.4 and 113.2
tweets per paper, respectively. With more than 10,000 unique Twitter users, PLOS ONE, BMJ, Nature,
Science, PNAS, NEMJ, JAMA, Lancet, Scientific Report, Nature Communications, JAMA Internal
Medicine, PLOS Biology, British Journal of Sports Medicine, Cell, Biotechnology Advances and BMJ
Open were tweeted by the largest audience on Twitter (Table 3). These journals reflect large
Handbook of Quantitative Science and Technology Research
20
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
multidisciplinary and biomedical journals with large readership, which have been previously identified
as highly tweeted, possibly because they exhibit a particular relevance to people’s everyday lives
(Costas et al., 2015b). Almost all of these journals maintain official Twitter accounts appearing among
the three most tweeting users, which suggests that Twitter has become an important platform for
journals and publishers to promote their contents. The number of unique Twitter users, in particular
for the general science journals seems to be low in comparison to global readership. For example,
while Nature claimed to have 3 million unique visitors per month (Nature Publishing Group, 2012),
as few as 42,365 Twitter users mentioned a 2015 paper. However, numbers seem to be in line with
print circulation (Nature Publishing Group, 2010).
Table 3 Twitter metrics for journals with the largest Twitter audience based on number of users
(≥10,000 users) including three most active users. Journal accounts are highlighted in bold.
Journal
Tw
itte
r
cov
era
ge
Tweets Users
N
Den
sity
Inte
nsi
ty
%R
Ts
N Top 3 most active users
(according to number of tweets;
official journal accounts are marked in bold)
PLOS ONE 70% 148,494 5.2 7.4 47% 59,210 aprendedorweb, uranus_2, PLOSONE
BMJ 91% 149,874 45.3 49.7 66% 57,622 bmj_latest, bookapharmacist, npceorg
Nature 100% 85,523 89.0 89.3 56% 42,365 Doyle_Media, randomshandom, TheRichardDoyle
Science 99% 78,141 70.0 70.8 67% 39,225 sciencemagazine, PedroArtino, rkeyserling
PNAS 89% 72,312 19.9 22.4 54% 36,496 abbrabot, EcoEvoJournals, uranus_2
NEMJ 99% 74,263 75.2 75.9 62% 33,142 medicineupdate, NEJM, JebSource
JAMA 95% 68,420 115.4 121.1 64% 31,018 JAMA_current, robarobberlover, ehlJAMA
Lancet 98% 45,752 69.9 71.6 65% 25,673 medicineupdate, darmtag, TheLancet
Scientific Reports 56% 41,177 4.6 8.3 50% 23,395 SciReports, uranus_2, geomatlab
Nature communications 78% 39,540 11.8 15.1 52% 21,274 NatureComms, Kochi_Study, PatrickGoymer
JAMA Internal Medicine 91% 48,882 99.8 110.1 69% 20,197 JAMAInternalMed, JAMA_current,
GeriatriaINNSZ
PLOS Biology 99% 24,481 90.0 91.3 70% 12,878 PLOSBiology, PLOS, nebiogroup
BJ of Sports Medicine 97% 26,121 75.7 77.7 69% 11,618 BJSM_BMJ, exerciseworks, SportScienceNI
Cell 96% 23,519 38.2 39.9 57% 11,081 Brianxbio, CellCellPress, topbiopapers
Biotechnology Advances 55% 14,823 113.2 205.9 67% 10,331 robinsnewswire, GrowKudos, ElsevierBiotech
BMJ Open 76% 16,882 11.5 15.1 60% 10,090 BMJ_Open, LS_Medical, SCPHRP
Going beyond peer-reviewed journals indexed in WoS, other sources (based on Altmetric metadata)
of scholarly documents are also frequently shared on Twitter. In fact, the largest number of tweeted
documents in dataset A came from arXiv: a total of 319,411 arXiv submissions were tweeted
1.1 million times by 110,134 users. As shown in Figure 2, the number of tweeted documents and
unique users derivate particularly for the most popular sources. Although arXiv, PLOS ONE and SSRN
are the most popular platforms according to the number of tweeted documents, Nature,
The Conversation and PLOS ONE are tweeted by the largest number of users.
The majority of popular sources are peer-reviewed journals indexed in WoS, which also lead the
ranking in dataset B (see above). Apart from these, links to repositories for documents (arXiv, SSRN,
bioRxiv) or data (figshare, Dryad) as well as to website like The Conversation or ClinicalTrials.gov
are also frequently tweeted. It should be mentioned that the document metadata in Altmetric is based
on a variety of sources and thus shows some inconsistencies. For example, the source is unknown for
6% of tweeted documents and 4% of tweets in dataset A, and some sources appear in various spellings.
Therefore, the number of distinct sources (49,379) or journal IDs (28,457) represents an overestimation
of different sources. These inconsistencies can also be found in the publication year and other
document metadata, which are essential to characterize what kind of scholarly output has been diffused
on Twitter.
Handbook of Quantitative Science and Technology Research
21
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Figure 2 Number of tweeted documents and unique users per source in dataset A (≥100 tweeted
documents).
1.3.2.2 Document characteristics
Besides disciplinary and topical differences, studies have also focused on determining what type of
documents are popular on Twitter. Papers with particular high Twitter visibility often had humorous
or entertaining contents rather than scientific merit (Haustein, Peters, Sugimoto, et al., 2014; Neylon,
2014). A study that coded title characteristics of 200 highly tweeted papers found that 16 included a
cultural reference (i.e., proverbs, idioms, fictional characters, music) and 13 were humorous or light
(Didegah, Bowman, Bowman, & Hartley, 2016). Bornmann (2014) reported that among papers
recommended on F1000, those labeled as good for teaching were frequently tweeted. Andersen and
Haustein (2015) found that meta-analysis and systematic reviews received significantly more tweets
than other medical study types.
Marking a clear distinction from citation patterns, particularly high Twitter activity was found for
document types that are usually considered uncitable (Haustein, Costas, et al., 2015). For example,
Twitter coverage for news items was twice as high as that for all documents. Twitter density was
highest for news items (3.0), editorial material (1.6) and reviews (1.4), by far exceeding the overall
average of 0.8 tweets per document. The success of these document types on Twitter suggests that
“documents that focus on topical subjects, debates and opinions, which are probably presented in
simpler and less technical language, are more likely to appear and become popular on Twitter”
(Haustein, Costas, et al., 2015, p. 8). Publications with shorter titles, fewer pages and references tend
to receive more tweets, while the opposite tendencies are usually observed for citations (Haustein,
Costas, et al., 2015).
1.3.2.3 Similarity to other types of usage
Since altmetrics were proposed as an alternative or complementary to traditional bibliometric
indicators, most studies analyze in how far these new impact metrics correlate with citations. The
motivation behind correlation studies lies in determining whether tweeting patterns are comparable to
citing behavior. Positive correlations between citation and tweets would indicate that tweets measure
something similar to but much earlier than citations, making tweet-based indicators predictors of
impact on the scholarly community (Eysenbach, 2011). Early studies argued that low or negative
correlation coefficients would indicate a different type of impact than that on citing authors, possibly
impact on society in general.
Handbook of Quantitative Science and Technology Research
22
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
The first study analyzing the relationship between tweets and citations found a significant association
between highly tweeted and highly cited papers published in the Journal of Medical Internet Research,
as highly tweeted publications identified 75% of those which were later highly cited (Eysenbach,
2011). However, this claim was based on 55 papers in a journal which itself maintained a strong Twitter
presence. For a set of 4,606 arXiv submissions, tweets were a better predictor of early citations than
downloads (Shuai et al., 2012). The generalizability of the finding that tweets were early indicators of
citation impact was later refuted by a large-scale analysis based on 1.3 million documents, which found
overall low correlations between tweets and citations (Haustein, Peters, Sugimoto, et al., 2014).
Correlations between tweets and citations vary between datasets due to particular differences between
disciplines or journals but are overall low between 0.1 and 0.2 (Barthel et al., 2015; Costas, Zahedi, &
Wouters, 2015a; Haustein, Costas, et al., 2015a; Haustein, Peters, Sugimoto, et al., 2014; Priem,
Piwowar, et al., 2012). It should be noted that correlations are affected by low Twitter coverage and
thus differ whether untweeted papers are excluded or included from the analysis (Haustein, Costas, et
al., 2015). Instead of replacing citations as a faster and better filter of relevant publications, as was
suggested in the altmetrics manifesto (Priem, Taraborelli, Groth, & Neylon, 2010), the number of
tweets seem to mirror visibility on other social media platforms, in particular Facebook, rather than
visibility within the community of citing authors (Barthel et al., 2015; Haustein, Costas, et al., 2015a).
If nothing else, the difference between tweet and citation counts as reflected in low correlations might
be due to the fundamental difference between the act of citing and tweeting (Haustein, Bowman, &
Costas, 2016).
A moderate negative correlation was found comparing publication output and tweeting activity of a
group of astrophysicists on Twitter, suggesting that researchers who tweet a lot focus their efforts on
communication and outreach rather than publishing peer-reviewed articles (Haustein, Bowman,
Holmberg, et al., 2014). This inverse relationship between a researchers’ standing in the scholarly
community and their visibility on Twitter has led to the so-called Kardashian index, a tongue-in-cheek
indicator that reveals that those who tweet more publish less and vice versa (Hall, 2014).
Rather than correlating tweets and citations Allen, Stanton, Di Pietro and Mosley (2013) aimed to
measure the effect of promoting articles on social media (including Twitter) on usage statistics.
Comparing the number of views, downloads and citations of randomly selected articles published in
PLOS ONE before and after promoting them on social media, article views and downloads increased
significantly but citations one year after publication and social media metrics did not.
1.3.3 How is scholarly output tweeted?
Not the least due to the evaluation community’s focus on counts, altmetrics research has focused much
less on tweet content than on correlations and other quantitative measures. Among those looking at
tweet content, the focus has been on the analysis of Twitter specific affordance use (Bowman, 2015b;
Weller et al., 2011). This includes in particular the use and analysis of hashtags, retweets and
@mentions, which are further described below.
Analyzing 270 tweets linking to journal articles, Thelwall et al. (2013) found that 42% contained the
title of the article, 41% summarized it briefly and 7% mentioned the author. As few as 5% explicitly
expressed interest in the article. While sentiment was absent in the great majority of tweets, 4% of
tweets were positive and none negative. Similarly a large-scale study that automatically identified
sentiments in tweets using SentiStrength found that the majority of tweets were neutral and that, if
sentiment was expressed, it was positive rather than negative (Friedrich, 2015). Based on
192,832 tweeted WoS documents published in 2012, 11% of 487,610 tweets were positive,
7% negative and 82% did not express any sentiment after removing the article title words from tweets.
Tweets linking to chemistry papers were the least likely to express sentiments (Friedrich, 2015;
Friedrich, Bowman, & Haustein, 2015).
Handbook of Quantitative Science and Technology Research
23
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
1.3.3.1 Retweets and @mentions
As described above, retweets and @mentions represent a particular form of conversational tweets,
which seemed to enjoy particular popularity among academic Twitter use. Half of the 4 million tweets
linking to 2015 WoS papers were retweets (Table 2), which suggests that a significant amount of
tweeting activity reflects information diffusion that does not involve much engagement. Compared to
the studies investigating retweet use among general Twitter users (boyd et al., 2010) and academics
(Haustein, Bowman, Holmberg, et al., 2014; Letierce et al., 2010; Priem & Costello, 2010; Work et
al., 2015), the share of retweets among tweeted journal articles is rather high. The percentage of
retweets tends to be lowest in disciplines with low Twitter coverage, which suggests that users in
disciplines with low Twitter uptake do not use it as much for information diffusion, possibly because
they are not as well connected. In 32 of 120 NSF specialties with DOI coverage above 50%, retweets
exceed original tweets (Table 2): retweeting was particularly common in Miscellaneous Zoology,
General & Internal Medicine, Miscellaneous Clinical Medicine and Ecology with retweet rates above
60% and low in Solid State Physics, Inorganic & Nuclear Chemistry, Chemical Physics and Applied
Chemistry with less than 20%.
1.3.3.2 Hashtags
Thirty-one percent of the 24.3 million tweets captured by Altmetric until June 2016 contained a
hashtag, which is comparable to other studies on hashtag use by academics (Haustein, Bowman,
Holmberg, et al., 2014; Work et al., 2015) but far higher than the 5% among a random sample of tweets
in 2009 (boyd et al., 2010). 401,287 unique hashtags were mentioned 12.6 million times, which
amounts to an average occurrence of 31 per unique term. 105,705 unique hashtags were used in tweets
linking to 2015 WoS papers. While 33% tweets contained at hashtag, 46% of all articles were described
with at least one hashtag. Each hashtag was mentioned on average 21 times for a total hashtag
frequency of 2.2 million. Hashtag frequency is extremely skewed, as 3% and 6% of hashtags are
responsible for 80% of hashtag occurrences in dataset A and B, respectively. For example, the most
popular hashtag in dataset A was used 162,754 times (1.3% of all occurrences), while 169,992 hashtags
only occurred once. Figure 3 demonstrates on a log-log scale the number of tweets hashtags were
mentioned in, as well as the number of distinct users mentioning each hashtag. While in general, a
linear relationship can be found between the number of occurrences and users, a few popular hashtags
are tweeted only by limited number of users, indicating a smaller community.
The most popular hashtags in dataset A were #science (Table 4), 1.3% of hashtag occurrence), #cancer
(0.9%), #physics (0.8%), #openaccess, #health (0.7%), #paper, #oa and #research (0.5% each). The
occurrence of #oa as well as #openaccess among the most frequent hashtags reflects the known
heterogeneity of folksonomies and the need for tag gardening when trying to analyze topics (Peters,
2009). WoS 2015 papers (dataset B) were most frequently tagged as #cancer (1.0%), #health,
#openaccess, #science (0.9%), #FOAMed, #Diabetes, #ornithology and #Psychiatry (0.6%). The order
changes, when considering the number of unique users instead of tweets per hashtag. Among hashtags
that occurred at least 1,000 times, the largest discrepancy between the number of tweets and users can
be observed for #genomeregulation (1,924 tweet; 10 users), #eprompt (2,281; 17) and #cryptocurrency
(4,515; 38), which were, on average, tweeted more than 100 times by the same users. On the contrary,
the user-hashtag ratio was lowest for #Fit (4,818; 4,743), #StandWithPP (Stand with Planned
Parenthood; 1,060; 972), #dataviz (1,010; 912), #coffee (1,517; 1,246), and #PWSYN (title of popular
science book The Patient Will See You Now; 1,017; 834), which indicates a widespread adoption
among Twitter users. Accordingly, these hashtags are more general and less scientific.
Handbook of Quantitative Science and Technology Research
24
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Figure 3 Number of users and number of tweets per hashtag for all tweets captured by Altmetric (A)
and for tweets to 2015 WoS papers (B).
Table 4 Hashtag statistics for datasets A and B with most frequent hashtags based on number of
tweets.
Most frequent
hashtags
Statistics of
hashtag frequency
Num
ber
of
twee
ts
Num
ber
of
use
rs
Use
rs p
er
has
hta
g
Num
ber
of
docu
men
ts
Num
ber
of
journ
als
Tw
eet
span
Dat
aset
A
#science
#cancer
#physics
#openaccess
#health
#paper
#oa
Mean 31 17 1 11
n/a
233
Standard deviation 617 237 4 209 459
Minimum 1 1 1 1 0
Maximum 162,754 65,334 1,425 54,845 1,825
99th percentile 395 229 6 129 1,746
90th percentile 20 15 2 8 1,036
75th percentile 5 4 1 2 182
50th percentile 2 2 1 1 0
Dat
aset
B
#cancer
#health
#openaccess
#science
#FOAMed
#Diabetes
#ornithology
Mean 21 13 1 7 4 102
Standard deviation 222 111 2 73 19 192
Minimum 1 1 1 1 1 0
Maximum 21,122 12,056 274 9,765 1,684 1,723
99th percentile 311 186 5 92 49 831
90th percentile 20 15 2 7 6 381
75th percentile 6 5 1 2 2 130
50th percentile 2 2 1 1 1 0
Handbook of Quantitative Science and Technology Research
25
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Table 4 shows hashtag-based stats for both datasets. As to be expected, hashtag frequency and the
number of unique terms is greater for dataset A, as it covers all documents in Altmetric and the whole
timespan, while dataset B is restricted to WoS documents published in 2015. On average, each hashtag
occurred in 21 tweets, was used by 13 users to tag 7 documents and 4 journals indexed in WoS. As
shown by the percentiles, hashtag occurrence is extremely skewed. On the individual level, the number
of users, documents and journals associated with a hashtag can provide information as to how general
and widespread a hashtag is, or how specific and relevant to only a small group of users. The timespan,
that is, the number of days between the first and last occurrence of a hashtag, indicates its topicality or
timeless relevance. For example, among hashtags that occurred at least 1,000 times, #diet, #water and
#nutrition were used during the course of more than four years to describe 2015 documents, while
#XmasBMJ lasted only 73 days. The first tweet linking to a 2015 WoS paper with the #diet hashtag
appeared 17 November 2011. The discrepancy between tweet date and publication date can be
described by the lag between online date and journal issue date (see Haustein et al. (2015b) for an
analysis of the publication date problematic).
1.3.4 When is scholarly output tweeted?
Tweet activity related to scholarly documents has been shown to occur shortly after publication and
disappear within a few days (Eysenbach, 2011; Shuai et al., 2012). Tweeted half-lives and delay
between publication and first tweet can thus be measured in hours rather than days. This short-lived
attention also points to Twitter being used to diffuse new papers instead of discussing them intensely.
It is, however, challenging to accurately calculate delay and decay for all publications in WoS as the
publication date of the journal does not sufficiently represent when a publication was actually
available. Even with the more accurate article-level information of online dates there are issues to
determine the actual date of publication, as demonstrated by tweets mentioning articles before they
were supposed to be published (Haustein, Bowman, et al., 2015b). Due to the inaccuracy of available
publication dates, tweet delay and tweeted half-lives are not computed.
Figure 4 Number of tweets per week day per year and mean of the weekly percentage per year.
As shown in Figure 4, there are clear differences between weekdays and weekends, reflecting patterns
of the work week, which has also been shown to have an effect on journal submissions (Cabanac &
Hartley, 2013) and download patterns (Wang et al., 2012). During the week, tweeting activity increases
from Monday (14% of tweets) to peak Wednesday (18%) and decrease again towards the weekend.
Twitter users tweet, on average, 23% more about scholarly documents on a Wednesday and 41% less
on a Sunday. Figure 4 also shows the different magnitude of Twitter activity among years, as well as
an overall increase throughout each year. While a general increase from January to December can be
observed for each of the four years of tweets, the general trend also reflects the academic year: activity
is higher in spring and fall and drops slightly in summer and particularly during the winter break during
the last two and first weeks of each year. Considering that Twitter activity often climaxes the day of
Handbook of Quantitative Science and Technology Research
26
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
or day after publication, the season and weekday of publication might influence a document’s visibility
on Twitter (Haustein, Bowman, et al., 2015b). Zhang and Paxson identified Twitter bots based on
tweeting patterns that were too regular to be human (Zhang & Paxson, 2011).
1.3.5 Where is scholarly output tweeted?
Twitter provides the possibility to geotag each tweet with precise latitude-longitude information of a
user’s current location. However, since this function is not activated by default, it is only rarely used.
Less than 5% of tweets contain geo coordinates (Graham, Hale, & Gaffney, 2014; Severo, Giraud, &
Pecout, 2015), which is why geotags of tweets are not a reliable source to determine where scholarly
output is tweeted. Another data source to determine the geographic distribution of Twitter users is to
analyze the location information provided in the Twitter bio. However, since the profile location is
usually not automatically generated but freely edited by users, it cannot be used without extensive data
cleaning. Takhteyec, Gruzd and Wellman (2012) showed that 8% of a sample of 3,360 Twitter profiles
contained specific latitude-longitude information, 57% named a location and 20% a country, while
15% used fictional places (e.g., Hogwarts) or too general descriptions to determine the users’
whereabouts. Using this profile information, Altmetric is able to determine location information for
two-thirds of its tweets. It becomes apparent that in many cases information is not accurate enough to
determine the exact location, as remote locations in the UK and Kansas are among the most frequent
tweet locations (Haustein & Costas, 2015a).
Table 5 Top 10 countries by number of users for dataset A and B.
Dataset A Dataset B Documents Tweets Users Documents Tweets Users
Number of
unique items 3,903,064 24,343,105 2,622,117 548,841 3,960,431 601,290
Some country
information 71% 58% 57% 70% 58% 58%
Missing country
information 69% 42% 44% 77% 42% 42%
To
p 1
0 c
oun
trie
s b
y
nu
mb
er o
f u
sers
US 33.0% 19.8% 20.1% US 36.2% 19.5% 19.4%
GB 21.6% 11.0% 8.3% GB 27.4% 12.3% 10.6%
CA 7.9% 2.9% 3.1% CA 9.3% 3.1% 3.4%
JP 4.3% 1.8% 2.3% ES 9.6% 3.3% 2.7%
AU 6.5% 2.9% 2.3% AU 7.6% 2.5% 2.4%
ES 7.5% 2.8% 2.2% JP 4.4% 1.5% 2.0%
FR 5.3% 1.5% 1.4% NL 4.0% 1.0% 1.3%
IN 2.6% 0.7% 1.1% FR 7.9% 1.6% 1.2%
NL 3.5% 1.0% 1.1% DE 5.6% 1.1% 0.9%
DE 4.1% 1.0% 0.9% IN 3.4% 0.7% 0.9%
Due to these limitations, the analysis of where users tweet scholarly documents is restricted to the
country level (Table 5). Altmetric provides location information for 57% of users, 58% of tweets and
71% of documents (dataset A) and 58% of both users and tweets and 70% of documents for the WoS
subset (dataset B). Users from the US are overrepresented as 20% of tweets are sent by Twitter users
with an identified location in the US, which is followed by the UK (8%), Canada (3%), Japan, Australia
and Spain (2% each). A similar distribution can be observed for the 2015 WoS articles (dataset B), as
the top 10 countries by number of users stay the same, although the UK, the Netherlands, Spain,
Canada and to a lesser extent Australia and Germany gain in percentage of users, while India, France,
Japan and the US lose in comparison to dataset A. While altmetrics have been marketed as
democratizers of science evaluation in the terms of having the potential to correct for biases created
by WoS and other US and English-centric journal databases, these results show that, when it comes to
Twitter, known biases persist or are even intensified on social media.
Handbook of Quantitative Science and Technology Research
27
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
1.3.6 Who tweets scholarly output?
One of the main motivations to consider tweets as an altmetric indicator is that Twitter is used by the
general public and thus, at least theoretically, offers insight into how non-academics engage with
scholarly output. In order to separate tweets by the public from those sent by members of the scholarly
community, Twitter users have to be identified and classified as such.
1.3.6.1 Identifying users who tweet scholarly content
One of the main challenges of determining the type of impact reflected by tweets to scientific papers
is to identify who is tweeting. While Mendeley provides certain standardized user demographics such
as academic status, discipline or country for users associated with a paper, the classification of tweets
by user type is restricted to Twitter bios. These self-descriptions are 160-character texts, which provide
users with the space to present themselves to other users of the microblogging platform.
Applying a codebook to determine who tweets scientific papers based on Twitter username, bio and
photo, a sample of 2,000 accounts tweeting links to articles published in Nature, PLOS ONE, PNAS
and Science, Tsou et al. (2015) found that almost one quarter of accounts were maintained by an
organization. Among these were mainly non-profits (42%), corporations (29%) and universities (13%),
while many were also classified as news, media or outreach institutions (19%). Among the
1,520 accounts identified as individuals, two thirds were male. One third of the users were identified
as having a PhD and 12% as students. This amounts to almost half of all identified individuals having
completed or pursuing a doctorate degree, which stands in glaring contrast to about 1% of the US
population with a PhD (Tsou et al., 2015), strongly suggesting that it is the scholarly community rather
than the general public who tweets links to scientific papers.
Applying a similar codebook to a random sample of 800 accounts tweeting 2012 WoS papers, 68% of
accounts were maintained by an individual, 21% by an organization, while 12% could not be identified
(Haustein et al., 2016), corroborating the findings by Tsou et al. (2015). Among individuals, 47% used
professional terms (e.g., doctor, MD, photographer) to describe themselves, 22% identified as
researchers (e.g., scientist, professor, postdoc), 13% as science communicators (e.g., writer, author,
journalist, blogger) and 7% as students (e.g., grad student, PhD candidate). Reflecting the blurred
boundaries between personal and professional communication, many individuals used words from
more than one of these categories to describe themselves. For example, 8% of accounts were classified
as researchers and professionals and 5% as professionals and science communicators (Haustein et al.,
2016). Science communicators were also the largest group of Twitter users mentioned by
astrophysicists (Holmberg et al., 2014). Although labor-intensive and based on little more than 160-
character self-descriptions, the above studies show that it is feasible to extract members of academia
from users tweeting scholarly documents. Keyword-based searches can be applied to identify scholars
in larger samples, but are limited by either low recall or low precision depending on the particular
query (Barthel et al., 2015).
It is considerably more challenging to identify members of the general public. Although many Twitter
bios contain terms depicting personal lives (e.g., father, wife, yoga lover), the presence of these terms
does not necessarily mean that accounts are maintained by non-academics, because scholars often
describe themselves in both personal and professional manner on Twitter (Bowman, 2015b; Haustein
& Costas, 2015b; Work et al., 2015). Similarly, it is challenging to distinguish members of the public
based on an indeterminate list of terms of non-academic professions (e.g., consultant, photographer),
especially when also considering accounts in languages other than English. Even the comparably
straight-forward identification of a researcher who strictly identifies as such on Twitter, becomes
problematic when they shared a paper out of private interests. For example, a tweet by a physicist
might actually reflect engagement by the public rather than scholarly communication, if they tweeted
about a cancer study as a member of a patient group rather than in their academic role.
Handbook of Quantitative Science and Technology Research
28
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
An alternative to classifying users based on publicly available information, is to approach them directly
and ask them who they are. Alperin (2015b) pioneered such a survey method on Twitter, which with
the help of an automated Twitter account, asked users who had tweeted a Scielo Brazil paper, whether
they were affiliated with a university. Such a direct approach might also be helpful to determine the
motivation of a user to tweet a specific paper, helping to quantify and distinguish different types of
tweets, such as endorsement or critical discusion, diffusion or self-promotion. Author self-citations or
self-tweets accounted for 7% of a sample of 270 tweets (Thelwall et al., 2013).
Table 6 Number of followers, tweets, tweet span, tweets per day and tweeting activity per week for
the most active users in dataset A (≥20,000 tweets per user).
The 24.3 million tweets captured by Altmetric were sent by 2.6 million users. Looking at the most
active users who tweeted more than 1,000 times during the whole period covered by Altmetric (Table
6), the presence of accounts automatically diffusing scholarly articles on Twitter becomes apparent
(Haustein, Bowman, Holmberg, et al., 2016). In fact, 15 of the 19 most productive accounts in Table
6 with more than 25,000 tweets self-identified bots (see below).
1.3.6.2 Classifying users by Twitter activity
Instead of classifying users according to their self-descriptions, accounts can also be grouped based on
their activity. Dividing Twitter accounts into three groups of top 1%, 9% and 90% of users (according
to number of tweets) helps to distinguish lead and highly active users from less active ones (Bruns &
Stieglitz, 2014). This classification provides insights into tweeting behavior of different types of users.
Separating the 601,290 users in dataset B by number of tweets linking to a 2015 WoS article, 6,016 lead
users, 54,535 highly active and 540,739 least active users can be identified. Lead users contributed
between 84 and 19,973 tweets, a median of 149 tweets per users, had on average 935 followers
(median; mean=3,862) and tweeted the 2015 papers during an average tweet span of 598 days. Highly
Twitter handle Number of
followers
Tweets Tweet
span
Tweets
per day
Tweeting activity per week
blackphysicists 12,914 92,583 1,826 50.7
MathPaper 1,889 73,239 1,086 67.4
anestesiaf 1,428 63,953 1,713 37.3
oceanologia 389 62,585 1,353 46.3
UIST_Papers20XX 35 50,211 1,360 36.9
UIST_Papers19XX 6 49,838 1,359 36.7
hiv_insight 14,328 48,714 1,822 26.7
russfeed 2,127 44,463 577 77.1
uranus_2 2,519 42,053 1,790 23.5
Immunol_papers 477 40,721 639 63.7
psych2evidence 513 40,658 410 99.2
InorganicNews 1,044 36,869 1,657 22.3
arXiv_trend 166 33,272 457 72.8
hlth_literacy 5,582 32,337 1,823 17.7
AstroPHYPapers 3,427 31,154 1,086 28.7
cirugiaf 406 30,692 1,710 17.9
ThihaSwe_dr 601 29,357 1,164 25.2
semantic_bot 0 28,908 28 1032.4
CondensedPapers 530 27,603 912 30.3
libroazuln 128 24,994 1,613 15.5
rnomics 1,520 24,905 1,821 13.7
PhysicsPaper 476 22,894 1,086 21.1
epigen_papers 788 21,916 739 29.7
Handbook of Quantitative Science and Technology Research
29
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
active users contributed between 9 and 83 tweets (median=16), had less followers (median=442.5;
mean=2,136) and shorter tweet spans (mean=388 days), while least active users tweeted up to eight
times (median=1), had 212 followers and were active for a period of 58 days.
Lead (top 1% of users), highly active (9%) and least active (90%) users contributed 43%, 31% and
25% of tweets to the entire set of 2015 WoS papers, respectively (Figure 5). Interestingly, these
percentages differ among NSF disciplines with least active users overrepresented among those
tweeting literature from Professional Fields, Social Sciences, Psychology and Earth and Space. On the
contrary, lead users were overrepresented in Chemistry, Physics, Mathematics and Engineering &
Technology, which were the fields exhibiting the lowest Twitter coverage, density and number of
unique users (Table 2). Assuming that the general public is least active when it comes to tweeting
about scholarly papers, they are more likely to engage with articles published in journals from the
Professional Fields and Social Sciences and less likely to tweet Chemical papers. The high presence
of lead users in Chemistry and Physics might, at least partly, be caused by accounts promoting these
papers automatically, such as @blackphysicists and @MathPaper, which were the two most active
accounts in dataset A, tweeting 51 and 67 scholarly documents per day (Table 6).
Figure 5 Percentage of tweets from lead users (1%), highly active users (9%) and least active users
(90%) per discipline (dataset B).
1.3.6.3 Twitter bots
Automated Twitter accounts have become prevalent on Twitter (Mowbray, 2014). About one fifth of
tweets sent during the 2016 presidential election were estimated to be sent by bots (Bessi & Ferrara,
2016) and almost one quarter of tweets in 2009 came from accounts tweeting more than 150 times per
day. Ferrara et al. (2016, p. 96) define social bots as “a computer algorithm that automatically produces
and interacts with humans on social media, trying to emulate and possibly alter their behavior”.
Automated Twitter accounts can be further distinguished between useful bots and antisocial or
spambots (Ferrara et al., 2016; Mowbray, 2014).
Bots are also infiltrating academic Twitter. Among a random sample of 800 Twitter accounts captured
by Altmetric, 8% seemed completely and 5% partially automated (Haustein et al., 2016), while
automated accounts who self-identified as such were responsible for 9% of tweets to arXiv submissions
(Haustein, Bowman, Holmberg, et al., 2016). Shuai et al. (2012) even removed half of the tweets to a
sample of tweets to documents on arXiv, as they were created by bots. Regardless of whether Twitter
accounts that automatically tweet scientific papers are considered useful or spam, it is safe to say that
their automated tweets do not reflect impact. In the context of altmetrics and tweets to scientific papers,
bot activity thus needs to be at least identified, if not entirely removed in an impact assessment.
Although spammers and excessive self-promotion was identified as a challenge by Altmetric, they still
considered gaming a rare and easy to identify threat in 2013 (Liu & Adie, 2013).
Handbook of Quantitative Science and Technology Research
30
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Twitter’s terms of service specifically address automation. While prohibiting spam, Twitter
encourages automated tweets if they “broadcast helpful information” (Twitter, 2017). However, what
is considered spam continuously evolves as users apply “new tricks and tactics” (Twitter, n.d.) to adapt
to or circumvent Twitter rules. Twitter bots can be identified based on specific regularities in their
tweeting behavior, such as the frequent and repetitive use of the same hashtags, URLs, tweet format
and content, regular temporal activity, as well as the follower/friends ratio, @mentions to non-
followers or account suspensions (Mowbray, 2014). Sixteen percent of Twitter accounts were
identified as automated based on not-uniform-enough or too-uniform tweeting patterns to be stemming
from a human (Zhang & Paxson, 2011). Social network indicators based on the Twitter follower and
friend network are considered more robust measures, as they are harder to influence (Yang,
Harkreader, & Gu, 2011). The BotOrNot algorithm additionally considers linguistic features and tweet
sentiment to detect automated Twitter accounts (Davis, Varol, Ferrara, Flammini, & Menczer, 2016).
However, as these spamming measures get known and integrated by Twitter to block spam accounts,
bots employ more sophisticated algorithms to avoid getting caught, resulting in an arms race between
those who create and those who seek to identify Twitter spam.
Figure 6 Query used to identify automated Twitter accounts.
Figure 7 6 Query used to identify publisher accounts.
Analyzing the most active users who have tweeted at least 1,000 times (dataset A), a keyword-based
query searching the Twitter bio as well as user name and handle revealed that among 2,043 accounts
248 identified themselves as automated (Figure 6) and 305 as journal or publisher accounts (Figure 7).
These make up 30% and 11% of the tweets sent by the 2,043 most active users (Table 7), which
correspond to 7% and 3% of the entire 24.3 million tweets to scholarly documents in dataset A. The
median number of followers is significantly lower than other accounts (Table 7) and as few as 6% of
the 1.8 million tweets sent by the 248 accounts contain @mentions. If scholarly bots do mention other
users, they often seem to reference journals such as @hiv_insight, which frequently mentions
@PLoSMedicine, @STI_BMJ and @JAMA_current. Other than social bots in the general
Twittersphere, scholarly bots seem to not try to emulate human behavior or game the system. They
rather resemble RSS feeds tweeting the paper title and a link, often specifying what type of information
they diffuse. Some even provide instructions to create similar feeds. For example, the Twitter bio of
@asthma_papers reads “RSS feed for #asthma papers in #Pubmed. Create a feed of your own using
instructions here: https://github.com/roblanf/phypapers”.
Handbook of Quantitative Science and Technology Research
31
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Table 7 Twitter metrics for self-identified bots, journal and publisher accounts and other accounts
based on users with at least 1,000 tweets (dataset A).
Most active Twitter accounts (≥1,000 tweets, n=2,043)
Followers Tweets Tweets
per day
Tweet
span
Self-identified bots n=248
30% of tweets
median 212 3,479 5.1 845
mean 1,014 7,339 14.4 923
std dev 2,781 10,390 67.0 477
min 0 1,001 0.6 28
max 25,003 73,239 1,032.4 1,823
Journal and
publisher accounts n=305
11% of tweets
median 3,199 1,670 1.2 1,647
mean 21,475 2,249 1.7 1,484
std dev 116,124 1,874 1.6 369
min 3 1,001 0.6 122
max 1,448,649 19,256 14.7 1,822
Other accounts n=1,490
59% of tweets
median 1,535 1,599 1.4 1,388
mean 5,236 2,408 2.9 1,278
std dev 14,091 3,670 9.3 496
min 0 1,000 0.6 8
max 228,224 92,583 297.3 1,826
Considering that scholars use Twitter to diffuse information and stay aware of relevant literature,
automated accounts might be considered useful. However, bots have shown to be harmful to society,
when they are used to influence public opinion and behavior such as political opinions or elections
(Bessi & Ferrara, 2016; Metaxas & Mustafaraj, 2012; Ratkiewicz, Conover, Meiss, Gonçalves,
Flammini, et al., 2011; Ratkiewicz, Conover, Meiss, Gonçalves, Patil, et al., 2011) or manipulation of
the stock market (Ferrara et al., 2016). If Twitter impact became part of the scholarly reward system,
Twitter bots might be able to similarly influence opinions or shape outcomes of certain research
metrics.
1.4 Conclusion and outlook This chapter provided an overview of the use of Twitter in scholarly communication. By demonstrating
who uses Twitter in academia and for what reasons, what types of scholarly outputs are diffused
how, where and when, it aimed to add context and help to interpret any scholarly metrics derived from
this and similar types of social media activity.
Research evaluators and managers were particularly excited at the prospect of an easily accessible data
source that would be able to capture traces of the societal impact of research. However, perhaps
unsurprisingly, the majority of tweets stem from stakeholders in academia rather than from members
of the general public, which indicates that the majority of tweets to scientific papers are more likely to
reflect scholarly communication rather than societal impact. At the same time, Twitter uptake in
academia lacks behind Twitter user by the general public. Twitter activity is influenced by
geographical and disciplinary biases and publication date. Known biases towards US and UK sources
persist, rather than democratizing scholarly communication and the reward system of science.
The majority of tweets linking to scientific articles appear shortly after their publication; tweeting half-
lives can be measured in hours rather than days. Moreover, one can observe weekday as well as
seasonal patterns with Twitter activity peaking Wednesdays and in the fall and plummeting during the
weekend and holiday season. Journal and publisher accounts as well as Twitter bots contribute
significantly to tweeting activity linked to academic papers, which suggests that a significant extent of
tweeting activity serves promotional purposes or is automated, which does neither reflect societal nor
scholarly impact. A large share of tweets contains hashtags and mention either the title or a short
summary of the paper they referred to. Half of all articles linking to 2015 WoS papers were retweets
Handbook of Quantitative Science and Technology Research
32
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
and the majority contained no sentiments. These tweeting characteristics emphasize particular low
engagement of users linking to journal articles. The main motivation for researchers to use Twitter is
information diffusion, networking and to stay up-to-date with the literature. However, the sheer brevity
of tweets makes intense discussions the exception rather than the rule on Twitter.
As citation behavior and motivations to cite or not cite certain sources are biased and influenced by
many factors other than a paper’s significance, not every citation represents impact. However, each
scholarly author is bound by scholarly norms to participate in the citation process. In some rare cases,
where scholars tweet corrections to publications or journals provide tweetable abstracts and organize
journal clubs, Twitter has started to be integrated in or even replace certain functions of formal journal
publishing. However, in most fields tweeting does not yet play an important role in scholarly
communication. In most disciplines, Twitter uptake is low and the platform is only used in a passive
or infrequent manner, or tweets reflect only a part of informal scholarly communication, such as
conference chatter. Moreover, Twitter uptake varies between disciplines, countries, journals, as well
as individuals and can be easily influenced and manipulated. The presence of automated Twitter
accounts which promote certain contents becomes particularly problematic when tweet counts become
the basis for measures of impact.
This is not to say that Twitter should be completely disregarded as a data source for scholarly metrics.
Rather, the microblogging platform should be approached critically in terms of what kind of use and
user populations it captures. By reviewing the role of Twitter in scholarly communication and
analyzing tweets linking to scholarly documents in depths and beyond crude counts, this chapter
attepmted to provide the basis for more sophisticated and well-balanced approaches to scholarly
Twitter metrics.
References Alhoori, H., & Furuta, R. (2014). Do altmetrics follow the crowd or does the crowd follow altmetrics? In
Digital Libraries (JCDL), 2014 IEEE/ACM Joint Conference on (pp. 375–378). IEEE. Retrieved from
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6970193
Allen, H. G., Stanton, T. R., Di Pietro, F., & Moseley, G. L. (2013). Social media release increases
dissemination of original articles in the clinical pain sciences. PloS One, 8(7), e68914–e68914.
https://doi.org/10.1371/journal.pone.0068914
Alperin, J. P. (2013). Ask not what altmetrics can do for you, but what altmetrics can do for developing
countries. Bulletin of the American Society for Information Science and Technology, 39(4), 18–21.
Alperin, J. P. (2015a). Geographic variation in social media metrics: an analysis of Latin American journal
articles. Aslib Journal of Information Management, 67(3), 289–304. https://doi.org/10.1108/AJIM-12-2014-
0176
Alperin, J. P. (2015b). Moving beyond counts: A method for surveying Twitter users. In altmetrics15: 5 years
in, what do we know? Amsterdam, The Netherlands. Retrieved from http://altmetrics.org/wp-
content/uploads/2015/09/altmetrics15_paper_3.pdf
Amir, M., Sampson, B. P., Endly, D., Tamai, J. M., Henley, J., Brewer, A. C., … Dellavalle, R. P. (2014).
Social networking sites: emerging and essential tools for communication in dermatology. JAMA
Dermatology, 150(1), 56–60. https://doi.org/10.1001/jamadermatol.2013.6340
Andersen, J. P., & Haustein, S. (2015). Influence of study type on Twitter activity for medical research papers.
In Proceedings of the 15th International Society of Scientometrics and Informetrics Conference (pp. 26–36).
Istanbul, Turkey. https://arxiv.org/abs/1507.00154
Association of American University Professors. (2013). Academic Freedom and Electronic Communications.
Retrieved from
https://www.aaup.org/file/Academic%20Freedom%20%26%20Electronic%20Communications.pdf
Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars ’
visibility on the social Web. In Proceedings of the 17th International Conference on Science and
Technology Indicators Montreal Canada 58 Sept 2012 (Vol. 52900, pp. 98–109). Montreal.
http://arxiv.org/abs/1205.5611
Handbook of Quantitative Science and Technology Research
33
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Barnes, N. G., & Lescault, A. M. (2013). College presidents out-blog and out-tweet corporate CEO’s. UMass
Center for Marketing Research. Retrieved from
http://www.umassd.edu/cmr/socialmediaresearch/collegepresidentsoutblog/
Barthel, S., Tönnies, S., Köhncke, B., Siehndel, P., & Balke, W.-T. (2015). What does Twitter Measure?
Influence of Diverse User Groups in Altmetrics. In Proceedings of the 15th ACM/IEEE-CE on Joint
Conference on Digital Libraries (pp. 119–128). Knoxville, Tennessee, USA: ACM Press.
https://doi.org/10.1145/2756406.2756913
Baym, M. (2014). The Perils and Pleasures of Tweeting with Fans. In K. Weller, A. Bruns, J. Burgess, M.
Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 221–236). New York: Peter Lang.
Bennett-Smith, M. (2013, June 4). Geoffrey Miller, Visiting NYU Professor, Slammed For Fat-Shaming
Obese PhD Applicants. Retrieved April 14, 2016, from
http://www.huffingtonpost.com/2013/06/04/geoffrey-miller-fat-shaming-nyu-phd_n_3385641.html
Bessi, A., & Ferrara, E. (2016). Social bots distort the 2016 U.S. Presidential election online discussion. First
Monday, 21(11). https://doi.org/10.5210/fm.v21i11.7090
Boateng, F., & Quan Liu, Y. (2014). Web 2.0 applications’ usage and trends in top US academic libraries.
Library Hi Tech, 32(1), 120–138. https://doi.org/10.1108/LHT-07-2013-0093
Bonetta, L. (2009). Should you be tweeting? Cell, 139(3), 452–453. https://doi.org/10.1016/j.cell.2009.10.017
Bornmann, L. (2014). Validity of altmetrics data for measuring societal impact: A study using data from
Altmetric and F1000Prime. Journal of Informetrics, 8(4), 935–950. https://doi.org/10.1016/j.joi.2014.09.007
Bornmann, L. (2016). Scientific Revolution in Scientometrics: the Broadening of Impact from Citation to
Societal. In C. R. Sugimoto (Ed.), Theories of Informetrics and Scholarly Communication (pp. 347–359).
Berlin: de Gruyter Mouton.
Bowman, T. D. (2015a). Differences in personal and professional tweets of scholars. Aslib Journal of
Information Management, 67(3), 356–371. https://doi.org/10.1108/AJIM-12-2014-0180
Bowman, T. D. (2015b, July). Investigating the use of affordances and framing techniques by scholars to
manage personal and professional impressions on Twitter (Dissertation). Indiana University, Bloomington,
IN, USA. Retrieved from http://www.tdbowman.com/pdf/2015_07_TDBowman_Dissertation.pdf
boyd, D., Golder, S., & Lotan, G. (2010). Tweet, Tweet, Retweet: Conversational Aspects of Retweeting on
Twitter (pp. 1–10). IEEE. https://doi.org/10.1109/HICSS.2010.412
Bruns, A., & Moe, H. (2014). Structural Layers of Communication on Twitter. In K. Weller, A. Bruns, J.
Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 15–28). New York: Peter Lang.
Bruns, A., & Stieglitz, S. (2014). Metrics for Understanding Communication on Twitter. In K. Weller, A.
Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 69–82). New York: Peter
Lang.
Bulger, M. E., Meyer, E. T., De la Flor, G., Terras, M., Wyatt, S., Jirotka, M., … Madsen, C. M. (2011).
Reinventing research? Information practices in the humanities (Research Information Network). Retrieved
from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1859267
Cabanac, G., & Hartley, J. (2013). Issues of work-life balance among JASIST authors and editors: Work-Life
Balance Issues Among JASIST Authors and Editors. Journal of the American Society for Information
Science and Technology, 64(10), 2182–2186. https://doi.org/10.1002/asi.22888
Cahn, P. S., Benjamin, E. J., & Shanahan, C. W. (2013). “Uncrunching” time: medical schools’ use of social
media for faculty development. Medical Education Online, 1, 1–6. https://doi.org/10.3402/meo.v18i0.20995
Carpenter, J., Wetheridge, L., Tanner, S., & Smith, N. (2012). Researchers of Tomorrow. Retrieved from
http://www.jisc.ac.uk/publications/reports/2012/researchers-of-tomorrow
Chaudhry, B. A., Glode, L. M., Gillman, M., & Miller, R. S. (2012). Trends in Twitter Use by Physicians at
the American Society of Clinical Oncology Annual Meeting , 2010 and 2011. Journal of Oncology Practice,
8(3), 173–178. https://doi.org/10.1200/JOP.2011.000483
Chretien, K. C., Azar, J., & Kind, T. (2011). Physicians on Twitter. JAMA, 305(6), 566–568.
https://doi.org/10.1001/jama.2011.68
Chu, S. K.-W., & Du, H. S. (2013). Social networking tools for academic libraries. Journal of Librarianship
and Information Science, 45(1), 64–75. https://doi.org/10.1177/0961000611434361
Cochran, A., Kao, L. S., Gusani, N. J., Suliburk, J. W., & Nwomeh, B. C. (2014). Use of Twitter to document
the 2013 Academic Surgical Congress. The Journal of Surgical Research, 190(1), 36–40.
https://doi.org/10.1016/j.jss.2014.02.029
Costas, R., Zahedi, Z., & Wouters, P. (2015a). Do “altmetrics” correlate with citations? Extensive comparison
of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for
Information Science and Technology, 66(10), 2003–2019. https://doi.org/10.1002/asi.23309
Handbook of Quantitative Science and Technology Research
34
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Costas, R., Zahedi, Z., & Wouters, P. (2015b). The thematic orientation of publications mentioned on social
media: Large-scale disciplinary comparison of social media metrics with citations. Aslib Journal of
Information Management, 67(3), 260–288. https://doi.org/10.1108/AJIM-12-2014-0173
Coverdale, A. (2011). Negotiating Doctoral Practices and Academic Identities Through the Adoption and Use
of Social and Participative Media. In Proceedings of the European Conference on eLearning 2011 (p. 909).
Brighton, UK.
Cronin, B. (2013a). Metrics à la mode. Journal of the American Society for Information Science and
Technology, 64(6), 1091–1091. https://doi.org/10.1002/asi.22989
Cronin, B. (2013b). The evolving indicator space (iSpace). Journal of the American Society for Information
Science and Technology, 64(8), 1523–1525. https://doi.org/10.1002/asi.23041
Darling, E. S., Shiffman, D., Côté, I., & Drew, J. A. (2013). The role of Twitter in the life cycle of a scientific
publication. Ideas in Ecology and Evolution, 6, 32–43. https://doi.org/10.7287/peerj.preprints.16v1
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016). BotOrNot: A System to Evaluate
Social Bots (pp. 273–274). ACM Press. https://doi.org/10.1145/2872518.2889302
Desai, T., Patwardhan, M., & Coore, H. (2014). Factors that contribute to social media influence within an
Internal Medicine Twitter learning community. F1000Research, 3, 120–120.
https://doi.org/10.12688/f1000research.4283.1
Desai, T., Shariff, A., Shariff, A., Kats, M., Fang, X., Christiano, C., & Ferris, M. (2012). Tweeting the
Meeting : An In-Depth Analysis of Twitter Activity at Kidney Week 2011. PLoS ONE, 7(7), 1–9.
https://doi.org/10.1371/journal.pone.0040253
Desrochers, N., Paul-Hus, A., Haustein, S., Costas, R., Mongeon, P., Quan-Haase, A., Bowman, T.D.,
Pecoskie, J., Tsou, A., & Larivière, V. (2018). Authorship, citations, acknowledgments and visibility in
social media: Symbolic capital in the multifaceted reward system of science.
Social Science Information, 1–26. https://doi.org/10.1177/0539018417752089 Didegah, F., Bowman, T. D., Bowman, S., & Hartley, J. (2016). Comparing the characteristics of highly cited
titles and highly alted titles. In 21st International Conference on Science and Technology Indicators (pp.
1190–1195). Valencia, Spain. Retrieved from
http://ocs.editorial.upv.es/index.php/STI2016/STI2016/paper/viewFile/4543/2327
Donelan, H. (2016). Social media for professional development and networking opportunities in academia.
Journal of Further and Higher Education, 40(5), 706–729. https://doi.org/10.1080/0309877X.2015.1014321
Duggan, M., Ellison, N. B., Lampe, C., Lenhart, A., & Madden, M. (2015, January 9). Demographics of Key
Social Networking Platforms. Retrieved from http://www.pewinternet.org/2015/01/09/demographics-of-key-
social-networking-platforms-2/
Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation
with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4), e123–e123.
https://doi.org/10.2196/jmir.2012
Fenner, M. (2013). What can article-level metrics do for you? PLoS Biology, 11(10), e1001687–e1001687.
https://doi.org/10.1371/journal.pbio.1001687
Ferguson, C., Inglis, S. C., Newton, P. J., Cripps, P. J. S., Macdonald, P. S., & Davidson, P. M. (2014). Social
media: A tool to spread information: A case study analysis of Twitter conversation at the Cardiac Society of
Australia & New Zealand 61st Annual Scientific Meeting 2013. Collegian, 21(2), 89–93.
https://doi.org/10.1016/j.colegn.2014.03.002
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016). The rise of social bots.
Communications of the ACM, 59(7), 96–104. https://doi.org/10.1145/2818717
Forkosh-Baruch, A., & Hershkovitz, A. (2012). A case study of Israeli higher-education institutes sharing
scholarly information with the community via social networks. The Internet and Higher Education, 15(1),
58–68. https://doi.org/10.1016/j.iheduc.2011.08.003
Friedrich, N. (2015, September 3). Applying sentiment analysis for tweets linking to scientific papers
(Bachelor’s thesis). Heinrich-Heine-Universität Düsseldorf, Düsseldorf.
Friedrich, N., Bowman, T. D., & Haustein, S. (2015). Do tweets to scientific articles contain positive or
negative sentiments? In altmetrics15: 5 years in, what do we know? Amsterdam, The Netherlands. Retrieved
from http://altmetrics.org/altmetrics15/friedrich/
Friedrich, N., Bowman, T. D., Stock, W. G., & Haustein, S. (2015). Adapting sentiment analysis for tweets
linking to scientific papers. In Proceedings of the 15th International Society of Scientometrics and
Informetrics Conference (pp. 107–108). Istanbul, Turkey. https://arxiv.org/abs/1507.01967
Gaffney, D., & Puschmann, C. (2014). Data Collection on Twitter. In K. Weller, A. Bruns, J. Burgess, M.
Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 55–67). New York: Peter Lang.
Handbook of Quantitative Science and Technology Research
35
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Gerber, A. (2012). Online trends from the first German trend study on science communication. In Science and
the Internet (pp. 13–18). Düsseldorf University Press.
Graham, M., Hale, S. A., & Gaffney, D. (2014). Where in the World Are You? Geolocation and Language
Identification in Twitter. The Professional Geographer, 66(4), 568–578.
https://doi.org/10.1080/00330124.2014.907699
Grajales, F. J., Sheps, S., Ho, K., Novak-Lauscher, H., & Eysenbach, G. (2014). Social media: a review and
tutorial of applications in medicine and health care. Journal of Medical Internet Research, 16(2), e13–e13.
https://doi.org/10.2196/jmir.2912
Grande, D., Gollust, S. E., Pany, M., Seymour, J., Goss, A., Kilaru, A., & Meisel, Z. (2014). Translating
Research For Health Policy: Researchers’ Perceptions And Use Of Social Media. Health Affairs, 33(7),
1278–1285. https://doi.org/10.1377/hlthaff.2014.0300
Greenwood, G. (2012). Examining the Presence of Social Media on University Web Sites. Journal of College
Admission, 216, 24–28.
Gruzd, A., Staves, K., & Wilk, A. (2012). Connected scholars: Examining the role of social media in research
practices of faculty using the UTAUT model. Computers in Human Behavior, 28(6), 2340–2350.
https://doi.org/10.1016/j.chb.2012.07.004
Gu, F., & Widén-Wulff, G. (2011). Scholarly communication and possible changes in the context of social
media. The Electronic Library, 29(6), 762–776. https://doi.org/10.1108/02640471111187999
Hadgu, A. T., & Jäschke, R. (2014). Identifying and analyzing researchers on twitter (pp. 23–32). ACM Press.
https://doi.org/10.1145/2615569.2615676
Hall, N. (2014). The Kardashian index: a measure of discrepant social media profile for scientists. Genome
Biology, 15(7), 424–424. https://doi.org/10.1186/s13059-014-0424-0
Hammarfelt, B. (2014). Using altmetrics for assessing research impact in the humanities. Scientometrics, 101,
1419–1430. https://doi.org/10.1007/s11192-014-1261-3
Harley, D., Acord, S. K., Earl-Novell, S., Lawrence, S., & King, C. J. (2010). Assessing the future landscape
of scholarly communication: an exploration of faculty values and needs in seven disciplines. Berkeley: The
Center for Studies in Higher Education, Univ Of California Press. Retrieved from
http://escholarship.org/uc/item/15x7385g
Harrington, S. (2014). Tweeting about the Telly: Live TV, Audiences and Social Media. In K. Weller, A.
Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 237–247). New York: Peter
Lang.
Haustein, S. (2016). Grand challenges in altmetrics: heterogeneity, data quality and dependencies.
Scientometrics. https://doi.org/10.1007/s11192-016-1910-9
Haustein, S., Bowman, T. D., & Costas, R. (2015a). “Communities of attention” around scientific
publications: who is tweeting about scientific papers? Presented at the Social Media & Society 2015
International Conference, Toronto, Canada.
Haustein, S., Bowman, T. D., & Costas, R. (2015b). When is an article actually published? An analysis of
online availability, publication, and indexation dates. In Proceedings of the 15th International Society of
Scientometrics and Informetrics Conference (pp. 1170–1179). Istanbul, Turkey.
https://arxiv.org/abs/1505.00796
Haustein, S., Bowman, T. D., & Costas, R. (2016). Interpreting “altmetrics”: viewing acts on social media
through the lens of citation and social theories. In C. R. Sugimoto (Ed.), Theories of Informetrics and
Scholarly Communication. A Festschrift in Honor of Blaise Cronin (pp. 372–405). Berlin: De Gruyter.
http://arxiv.org/abs/1502.05701
Haustein, S., Bowman, T. D., Holmberg, K., Peters, I., & Larivière, V. (2014). Astrophysicists on Twitter: An
in-depth analysis of tweeting and scientific publication behavior. Aslib Journal of Information Management,
66(3), 279–296. https://doi.org/10.1108/AJIM-09-2013-0081
Haustein, S., Bowman, T. D., Holmberg, K., Tsou, A., Sugimoto, C. R., & Larivière, V. (2016). Tweets as
impact indicators: Examining the implications of automated “bot” accounts on Twitter. Journal of the
Association for Information Science and Technology, 67(1), 232–238. https://doi.org/10.1002/asi.23456
Haustein, S., Bowman, T. D., Macaluso, B., Sugimoto, C. R., & Larivière, V. (2014). Measuring Twitter
activity of arXiv e-prints and published papers. In altmetrics14: expanding impacts and metrics, Workshop
at Web Science Conference 2014. https://doi.org/10.6084/m9.figshare.1041514%20
Haustein, S., & Costas, R. (2015a). Determining Twitter audiences: Geolocation and number of followers. In
altmetrics15: 5 years in, what do we know? Amsterdam, The Netherlands. Retrieved from
http://altmetrics.org/altmetrics15/haustein/
Handbook of Quantitative Science and Technology Research
36
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Haustein, S., & Costas, R. (2015b). Identifying Twitter audiences: who is tweeting about scientific papers?
Presented at the Metrics 2015 ASIS&T SIG/MET Workshop. Retrieved from
https://www.asist.org/SIG/SIGMET/wp-content/uploads/2015/10/sigmet2015_paper_11.pdf
Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The
effect of document properties and collaboration patterns. PLoS ONE, 10(3), e0120495.
https://doi.org/10.1371/journal.pone.0120495
Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (2014). Tweets vs. Mendeley readers: How
do these two social media metrics differ? It - Information Technology, 56(5), 207–215.
https://doi.org/10.1515/itit-2014-1048
Haustein, S., Minik, V., Brinson, D., Hayes, E., Costas, R., & Sugimoto, C. R. (2016). Identifying Twitter user
communities in the context of altmetrics. In 3rd Altmetrics Conference 3:AM. Bucharest, Romania.
Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014). Coverage and adoption of
altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163.
https://doi.org/10.1007/s11192-013-1221-3
Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2014). Tweeting biomedicine: an
analysis of tweets and citations in the biomedical literature. Journal of the Association for Information
Science and Technology, 65(4), 656–669. https://doi.org/10.1002/asi.23101
Haustein, S., Sugimoto, C. R., & Larivière, V. (2015). Guest editorial: social media in scholarly
communication. Aslib Journal of Information Management, 67(3). https://doi.org/10.1108/AJIM-03-2015-
0047
Hawkins, C. M., Duszak, R., & Rawson, J. V. (2014). Social media in radiology: early trends in Twitter
microblogging at radiology’s largest international meeting. Journal of the American College of Radiology :
JACR, 11(4), 387–90. https://doi.org/10.1016/j.jacr.2013.07.015
Herman, B. (2014, September 5). Steven Salaita Twitter Scandal: University Offers Settlement, But Free
Speech Questions Linger. International Business Times. Retrieved from http://www.ibtimes.com/steven-
salaita-twitter-scandal-university-offers-settlement-free-speech-questions-linger-1678854
Holmberg, K. (2014). The impact of retweeting. Figshare. Retrieved from
http://dx.doi.org/10.6084/m9.figshare.1037555
Holmberg, K., Bowman, T. D., Haustein, S., & Peters, I. (2014). Astrophysicists’ conversational connections
on twitter. PloS One, 9(8), e106086–e106086. https://doi.org/10.1371/journal.pone.0106086
Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication.
Scientometrics, 101(2), 1027–1042. https://doi.org/10.1007/s11192-014-1229-3
Honeycutt, C., & Herring, S. C. (2009). Beyond Microblogging: Conversation and Collaboration via Twitter
(pp. 1–10). IEEE. https://doi.org/10.1109/HICSS.2009.89
Hussain, A. (2015). Adoption of Web 2.0 in library associations in the presence of social media. Program,
49(2), 151–169. https://doi.org/10.1108/PROG-02-2013-0007
Ingeno, L. (2013, June 4). Fat-Shaming in Academe. Inside Higher Ed. Retrieved from
https://www.insidehighered.com/news/2013/06/04/outrage-over-professors-twitter-post-obese-students
Jalali, A., & Wood, T. (2013). Analyzing Online Impact of Canadian Conference of Medical Education
through Tweets. Education in Medicine Journal, 5(3). https://doi.org/10.5959/eimj.v5i3.162
Java, A., Song, X., Finin, T., & Tseng, B. (2007). Why we twitter: understanding microblogging usage and
communities (pp. 56–65). ACM Press. https://doi.org/10.1145/1348549.1348556
Kamel Boulos, M. N., & Anderson, P. F. (2012). Preliminary survey of leading general medicine journals’ use
of Facebook and Twitter. Journal of the Canadian Health Libraries Association, 33(2), 38–47.
Ke, Q., Ahn, Y.-Y., & Sugimoto, C. R. (2017). A systematic identification and analysis of scientists on
Twitter. PLOS ONE, 12(4), e0175368. https://doi.org/10.1371/journal.pone.0175368
Kelly, R. (2009). Twitter study (White Paper). San Antonio, TX: Pear Analytics. Retrieved from
http://pearanalytics.com/wp-content/uploads/2012/12/Twitter-Study-August-2009.pdf
Kimmons, R., Veletsianos, G., & Woodward, S. (2017). Institutional Uses of Twitter in U.S. Higher
Education. Innovative Higher Education, 42(2), 97–111. https://doi.org/10.1007/s10755-016-9375-6
Knight, S. R. (2014). Social media and online attention as an early measure of the impact of research in solid
organ transplantation. Transplantation, 98(5), 490–6. https://doi.org/10.1097/TP.0000000000000307
Kortelainen, T., & Katvala, M. (2012). “Everything is plentiful—Except attention”. Attention data of
scientific journals on social web tools. Journal of Informetrics, 6(4), 661–668.
https://doi.org/10.1016/j.joi.2012.06.004
Letierce, J., Passant, A., Breslin, J., & Decker, S. (2010). Understanding how Twitter is used to spread
scientific messages. Retrieved from http://journal.webscience.org/314/
Handbook of Quantitative Science and Technology Research
37
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Leung, E. Y. L., Siassakos, D., & Khan, K. S. (2015). Journal Club via social media: authors take note of the
impact of #BlueJC. BJOG: An International Journal of Obstetrics & Gynaecology, 122(8), 1042–1044.
https://doi.org/10.1111/1471-0528.13440
Linvill, D. L., McGee, S. E., & Hicks, L. K. (2012). Colleges’ and universities’ use of Twitter: A content
analysis. Public Relations Review, 38(4), 636–638. https://doi.org/10.1016/j.pubrev.2012.05.010
Liu, J., & Adie, E. (2013). Five Challenges in Altmetrics : A Toolmaker’ s Perspective. Bulletin of
Associatioon for Infomation Science and Technology, 39(4), 31–34.
Loeb, S., Bayne, C. E., Frey, C., Davies, B. J., Averch, T. D., Woo, H. H., … Eggener, S. E. (2014). Use of
social media in urology: data from the American Urological Association (AUA). BJU International, 113(6),
993–998. https://doi.org/10.1111/bju.12586
Lulic, I., & Kovic, I. (2013). Analysis of emergency physicians’ Twitter accounts. Emergency Medicine
Journal : EMJ, 30(5), 371–6. https://doi.org/10.1136/emermed-2012-201132
Maleki, A. (2014). Networking in science tweets linking to scholarly articles of Iran. Presented at the
SIG/MET Workshop, ASIS&T 2014 Annual Meeting, Seattle. Retrieved from
http://www.asis.org/SIG/SIGMET/data/uploads/sigmet2014/maleki.pdf
Mccrea, P. (2011). Reflections on Academic Blogging as a Vehicle for Professional Development. In
Proceedings of the European Conference on eLearning (p. 997). Brighton, UK.
McKendrick, D. R., Cumming, G. P., & Lee, A. J. (2012). Increased Use of Twitter at a Medical Conference:
A Report and a Review of the Educational Opportunities. Journal of Medical Internet Research, 14(6), e176.
https://doi.org/10.2196/jmir.2144
Mehta, N., & Flickinger, T. (2014). The Times They Are A-Changin’: Academia, Social Media and the JGIM
Twitter Journal Club. Journal of General Internal Medicine, 29(10), 1317–1318.
https://doi.org/10.1007/s11606-014-2976-9
Metaxas, P. T., & Mustafaraj, E. (2012). Social Media and the Elections. Science, 338(6106), 472–473.
https://doi.org/10.1126/science.1230456
Micieli, R., & Micieli, J. a. (2012). Twitter as a tool for ophthalmologists. Canadian Journal of
Ophthalmology. Journal Canadien D’ophtalmologie, 47(5), 410–3.
https://doi.org/10.1016/j.jcjo.2012.05.005
Miller, G. (2011). Social Scientists Wade Into The Tweet Stream. Science, 333(6051), 1814–15.
https://doi.org/10.1126/science.333.6051.1814
Miller, V. (2008). New Media, Networking and Phatic Culture. Convergence: The International Journal of
Research into New Media Technologies, 14(4), 387–400. https://doi.org/10.1177/1354856508094659
Mishori, R., Levy, B., & Donvan, B. (2014). Twitter Use at a Family Medicine Conference : Analyzing
#STFM13. Family Medicine, 46(8).
Mou, Y. (2014). Presenting professorship on social media: from content and strategy to evaluation. Chinese
Journal of Communication, 7(4), 389–408. https://doi.org/10.1080/17544750.2014.938669
Mowbray, M. (2014). Automated Twitter Accounts. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C.
Puschmann (Eds.), Twitter and Society (pp. 183–94). New York: Peter Lang.
Nason, G. J., O’Kelly, F., Kelly, M. E., Phelan, N., Manecksha, R. P., Lawrentschuk, N., & Murphy, D. G.
(2015). The emerging use of Twitter by urological journals: Use of Twitter by urological journals. BJU
International, 115(3), 486–490. https://doi.org/10.1111/bju.12840
Nature Publishing Group (2010). Media Options. Retrieved from:
https://web.archive.org/web/20110513233735/http://www.nature.com/advertising/resources/pdf/2010nature
mediakit.pdf?prod_code=NATURE&prod=Nature
Nature Publishing Group. (2012). Announcement: A new iPad app for Nature readers. Nature, 492(7428),
154–154. https://doi.org/10.1038/492154a
Nature Publishing Group. (2015). Author Insights 2015 survey. Figshare.
Nentwich, M. (2011). Das Web 2.0 in der wissenschaftlichen Praxis. In Digitale
Wissenschaftskommunikation–Formate und ihre Nutzung (pp. 35–53). Giessen. Retrieved from
http://geb.uni-giessen.de/geb/volltexte/2011/8227/pdf/DigitaleWissenschaftskommunikation_2011.pdf
Neylon, C. (2014, October 3). Altmetrics: What are they good for? Retrieved January 10, 2015, from
http://blogs.plos.org/opens/2014/10/03/altmetrics-what-are-they-good-for/
Osterrieder, A. (2013). The value and use of social media as communication tool in the plant sciences. Plant
Methods, 9(1), 26–26. https://doi.org/10.1186/1746-4811-9-26
Pasquini, L. A., & Evangelopoulos, N. (2015). Organizational identity, meaning, and values: analysis of social
media guideline and policy documents (pp. 1–5). ACM Press. https://doi.org/10.1145/2789187.2789198
Handbook of Quantitative Science and Technology Research
38
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Paßmann, J., Boeschoten, T., & Schäfer, M. T. (2014). The gift of the gab. Retweet cartels and gift economies
on Twitter. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp.
331–344). New York: Peter Lang.
Pearce, N., Weller, M., Scanlon, E., & Ashleigh, M. (2010). Digital Scholarship Considered: How New
Technologies Could Transform Academic Work. In Education, 16(1), 33–44.
Peters, I. (2009). Folksonomies indexing and retrieval in Web 2.0. Berlin, Allemagne: De Gruyter/Saur.
Pomerantz, J., Hank, C., & Sugimoto, C. R. (2015). The State of Social Media Policies in Higher Education.
PLOS ONE. https://doi.org/10(5): e0127485. doi:10.1371
Ponte, D., & Simon, J. (2011). Scholarly Communication 2.0: Exploring Researchers’ Opinions on Web 2.0
for Scientific Knowledge Creation, Evaluation and Dissemination. Serials Review, 37(3), 149–156.
https://doi.org/10.1016/j.serrev.2011.06.002
Prabhu, V., & Rosenkrantz, A. B. (2015). Enriched Audience Engagement Through Twitter: Should More
Academic Radiology Departments Seize the Opportunity? Journal of the American College of Radiology,
12(7), 756–759. https://doi.org/10.1016/j.jacr.2015.02.016
Priem, J. (2014). Altmetrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: harnessing
multidimensional indicators of performance (pp. 263–287). Cambridge, MA: MIT Press.
Priem, J., Costello, K., & Dzuba, T. (2012). Prevalence and use of Twitter among scholars. Presented at the
Metrics 2011 Symposium on Informetric and Scientometric Research.
https://doi.org/http://dx.doi.org/10.6084/m9.figshare.104629
Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the American
Society for Information Science and Technology, 47(1), 1–4. https://doi.org/10.1002/meet.14504701201
Priem, J., Piwowar, H., & Hemminger, B. M. (2012). Altmetrics in the wild: Using social media to explore
scholarly impact. arXiv Print, 1–17.
Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010, October 26). Altmetrics: A manifesto. Retrieved
from http://altmetrics.org/manifesto/
Procter, R. N., Williams, R., Stewart, J., Poschen, M., Snee, H., Voss, A., & Asgari-Targhi, M. (2010). If you
build it, will they come? How researchers perceive and use Web 2.0. London, UK: Research Network
Information. Retrieved from http://wrap.warwick.ac.uk/56246
Pscheida, D., Albrecht, S., Herbst, S., Minet, C., & Köhler, T. (2013). Nutzung von Social Media und
onlinebasierten Anwendungen in der Wissenschaft. ZBW–Deutsche Zentralbibliothek für
Wirtschaftswissenschaften–Leibniz-Informationszentrum Wirtschaft. Retrieved from
http://www.qucosa.de/fileadmin/data/qucosa/documents/13296/Science20_Datenreport_2013_PDF_A.pdf
Puschmann, C. (2014). Opening Science. Opening Science, 89–106.
https://doi.org/10.1007/978-3-319-00026-8
Ratkiewicz, J., Conover, M. D., Meiss, M., Gonçalves, B., Flammini, A., & Menczer, F. (2011). Detecting and
tracking political abuse in social media. In Proceedings of the Fifth International AAAI Conference on
Weblogs and Social Media (pp. 297–304).
Ratkiewicz, J., Conover, M., Meiss, M., Gonçalves, B., Patil, S., Flammini, A., & Menczer, F. (2011). Truthy:
mapping the spread of astroturf in microblog streams (p. 249). ACM Press.
https://doi.org/10.1145/1963192.1963301
Reinhardt, W., Ebner, M., Beham, G., & Costa, C. (2009). How People are using Twitter during Conferences.
In Creativity and Innovation Competencies on the Web, Proceeding of 5th EduMedia conference (pp. 145–
156). Salzburg: Hornung-Prähauser, V., Luckmann, M. (Ed.). Retrieved from http://lamp.tu-
graz.ac.at/~i203/ebner/publication/09_edumedia.pdf
Robinson-García, N., Torres-Salinas, D., Zahedi, Z., & Costas, R. (2014). New data, new possibilities:
Exploring the insides of Altmetric.com. El Profesional de La Información, 23(4), 359–366.
https://doi.org/10.3145/epi.2014.jul.03
Rogers, R. (2014). Debanalising Twitter. The Transformation of an Object of Study. In K. Weller, A. Bruns, J.
Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. ix–xxvi). New York: Peter Lang.
Rothschild, S., & Unglesbee, B. (2013, September). Kansas University professor receiving death threats over
NRA tweet. The Dispatch. Retrieved from http://www.shawneedispatch.com/news/2013/sep/24/kansas-
university-professor-receiving-death-threat/
Rowlands, I., Nicholas, D., Russell, B., Canty, N., & Watkinson, A. (2011). Social media use in the research
workflow. Learned Publishing, 24(3), 183–195. https://doi.org/10.1087/20110306
Schmidt, J.-H. (2014). Twitter and the Rise of Personal Publics. In K. Weller, A. Bruns, J. Burgess, M. Mahrt,
& C. Puschmann (Eds.), Twitter and Society (pp. 3–12). New York: Peter Lang.
Handbook of Quantitative Science and Technology Research
39
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Seglen, P. O. (1997). Citations and journal impact factors: questionable indicators of research quality. Allergy,
52(11), 1050–1056. https://doi.org/10.1111/j.1398-9995.1997.tb00175.x
Severo, M., Giraud, T., & Pecout, H. (2015). Twitter data for urban policy making: an analysis on four
European cities. In C. Levallois (Ed.), Handbook of Twitter for Research. EMLYON.
Sharma, N. K., Ghosh, S., Benevenuto, F., Ganguly, N., & Gummadi, K. (2012). Inferring who-is-who in the
Twitter social network (p. 55). ACM Press. https://doi.org/10.1145/2342549.2342563
Shuai, X., Pepe, A., & Bollen, J. (2012). How the scientific community reacts to newly submitted preprints:
article downloads, Twitter mentions, and citations. PloS One, 7(11), e47523–e47523.
https://doi.org/10.1371/journal.pone.0047523
Shulman, J., Yep, J., & Tomé, D. (2015). Leveraging the Power of a Twitter Network for Library Promotion.
The Journal of Academic Librarianship, 41(2), 178–185. https://doi.org/10.1016/j.acalib.2014.12.004
Sopan, A., Rey, P. J., Butler, B., & Shneiderman, B. (2012). Monitoring Academic Conferences: Real-Time
Visualization and Retrospective Analysis of Backchannel Conversations. 2012 International Conference on
Social Informatics, (SocialInformatics), 62–69. https://doi.org/10.1109/SocialInformatics.2012.20
Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics:
a review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037–
2062. https://doi.org/10.1002/asi.23833
Takhteyev, Y., Gruzd, A., & Wellman, B. (2012). Geography of Twitter networks. Social Networks, 34(1),
73–81. https://doi.org/10.1016/j.socnet.2011.05.006
Taylor, M. (2013). Towards a common model of citation : some thoughts on merging altmetrics and
bibliometrics. Research Trends, (35), 1–6.
Tenopir, C., Volentine, R., & King, D. W. (2013). Social media and scholarly reading. Online Information
Review, 37(2), 193–216. https://doi.org/10.1108/OIR-04-2012-0062
Thangasamy, I. A., Leveridge, M., Davies, B. J., Finelli, A., Stork, B., & Woo, H. H. (2014). International
Urology Journal Club via Twitter: 12-Month Experience. European Urology, 66(1), 112–117.
https://doi.org/10.1016/j.eururo.2014.01.034
Thelwall, M., Tsou, A., Weingart, S., Holmberg, K., & Haustein, S. (2013). Tweeting Links to Academic
Articles. Cybermetrics: International Journal of Scientometrics, Informetrics and Bibliometrics, 17(1), 1–8.
Topf, J. M., & Hiremath, S. (2015). Social media, medicine and the modern journal club. International Review
of Psychiatry, 27(2), 147–154. https://doi.org/10.3109/09540261.2014.998991
Tsou, A., Bowman, T. D., Ghazinejad, A., & Sugimoto, C. R. (2015). Who tweets about science? In
Proceedings of the 2015 International Society for Scientometrics and Informetrics (pp. 95–100). Istanbul,
Turkey. Retrieved from
https://pdfs.semanticscholar.org/81fe/8b63188cf25648a7c592bc6b5457fee3c101.pdf?_ga=1.184338726.126
4550827.1478885332
Twitter. (2017, April 6). Automation rules. Retrieved from https://support.twitter.com/articles/76915
Twitter. (n.d.). The Twitter Rules. Retrieved from https://support.twitter.com/articles/18311
Van Noorden, R. (2013). Brazilian citation scheme outed. Nature, 500(7464), 510–511.
https://doi.org/10.1038/500510a
Van Noorden, R. (2014). Online collaboration: Scientists and the social network. Nature, 512(7513), 126–129.
https://doi.org/10.1038/512126a
Veletsianos, G. (2012). Higher education scholars’ participation and practices on Twitter. Journal of
Computer Assisted Learning, 28(4), 336–349. https://doi.org/10.1111/j.1365-2729.2011.00449.x
Veletsianos, G. (2013). Open practices and identity: Evidence from researchers and educators’ social media
participation. British Journal of Educational Technology, 44(4), 639–651. https://doi.org/10.1111/bjet.12052
Verishagen, N., & Hank, C. (2014). Are there birds in the library? The extent of Twitter adoption and use by
Canadian academic libraries. First Monday, 19(11). https://doi.org/10.5210/fm.v19i11.4945
Wang, X., Xu, S., Peng, L., Wang, Z., Wang, C., Zhang, C., & Wang, X. (2012). Exploring scientists’
working timetable: Do scientists often work overtime? Journal of Informetrics, 6(4), 655–660.
https://doi.org/10.1016/j.joi.2012.07.003
Warzel, C. (2014, April 15). Meet The Man Behind Twitter’s Most Infamous Phrase. BuzzFeed News.
Retrieved from https://www.buzzfeed.com/charliewarzel/meet-the-man-behind-twitters-most-infamous-
phrase?utm_term=.jwWokRoZw#.fnElNylG4
Weller, K., Bruns, A., Burgess, J., Mahrt, M., & Puschmann, C. (2014). Twitter and Society: An Introduction.
In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. xxiv–
xxxviii). New York: Peter Lang.
Handbook of Quantitative Science and Technology Research
40
Glänzel, Moed, Schmoch, & Thelwall (Eds.)
Weller, K., Dornstädter, R., Freimanis, R., Klein, R. N., & Perez, M. (2010). Social software in academia:
Three studies on users’ acceptance of Web 2.0 Services. Retrieved from
http://journal.webscience.org/360/2/websci10_submission_62.pdf
Weller, K., Dröge, E., & Puschmann, C. (2011). Citation Analysis in Twitter : Approaches for Defining and
Measuring Information Flows within Tweets during Scientific Conferences. In Proceedings of the
ESWC2011 Workshop on “Making Sense of Microposts”: Big things come in small packages (Vol. 718, pp.
1–12). Heraklion, Greece. Retrieved from http://ceur-ws.org/Vol-718/msm2011_proceedings.pdf
Weller, K., & Puschmann, C. (2011). Twitter for Scientific Communication : How Can Citations / References
be Identified and Measured ? In Proceedings of the ACM WebSci’11. Koblenz, Germany.
Whitburn, T., Walshe, C., & Sleeman, K. E. (2015). International Palliative Care Journal Club on Twitter:
Experience so far. BMJ Supportive & Palliative Care, 5(1), 120.2-120. https://doi.org/10.1136/bmjspcare-
2014-000838.48
White, H. D. (2000). Towards ego-centered citation analysis. In B. Cronin & H. B. Atkins (Eds.), The Web of
Knowledge: A Festschrift in Honor of Eugene Garfield (pp. 475–496). Medford, NJ: Information Today.
White, H. D. (2001). Authors as citers over time. Journal of the American Society for Information Science and
Technology, 52(2), 87–108.
https://doi.org/10.1002/1097-4571(2000)9999:9999<::AID-ASI1542>3.0.CO;2-T
Wilkinson, C., & Weitkamp, E. (2013). A case study in serendipity: environmental researchers use of
traditional and social media for dissemination. PloS One, 8(12), e84339–e84339.
https://doi.org/10.1371/journal.pone.0084339
Wilson, M. W., & Starkweather, S. (2014). Web Presence of Academic Geographers: A Generational Divide?
The Professional Geographer, 66(1), 73–81. https://doi.org/10.1080/00330124.2013.765290
Woolston, C. (2015). Potential flaws in genomics paper scrutinized on Twitter. Nature, 7553(521), 397.
https://doi.org/10.1038/521397f
Work, S., Haustein, S., Bowman, T. D., & Larivière, V. (2015). Social Media in Scholarly Communication. A
Review of the Literature and Empirical Analysis of Twitter Use by SSHRC Doctoral Award Recipients
(Study commissioned by the Social Sciences and Humanities Research Council).
Yang, C., Harkreader, R. C., & Gu, G. (2011). Die Free or Live Hard? Empirical Evaluation and New Design
for Fighting Evolving Twitter Spammers. In R. Sommer, D. Balzarotti, & G. Maier (Eds.), Recent Advances
in Intrusion Detection (Vol. 6961, pp. 318–337). Berlin, Heidelberg: Springer Berlin Heidelberg.
https://doi.org/10.1007/978-3-642-23644-0_17
Zedda, M., & Barbaro, A. (2015). Adoption of web 2.0 tools among STM publishers. How social are scientific
journals? Journal of the EAHIL, 11(1), 9–12.
Zhang, C. M., & Paxson, V. (2011). Detecting and analyzing automated activity on Twitter. In Proceedings of
the 12th International Conference on Passive and Active Measurement PAM’11 (pp. 102–111). Berlin,
Heidelberg: Springer. Retrieved from http://www.icir.org/vern/papers/pam11.autotwit.pdf
Zhao, D., & Rosson, M. B. (2009). How and why people Twitter: the role that micro-blogging plays in
informal communication at work (p. 243). ACM Press. https://doi.org/10.1145/1531674.1531710