Abstract
SUMMARY: Social media use by professional organizations has increased as a platform to disseminate information, affording an alternative avenue to engage membership and the public. The American Journal of Neuroradiology (AJNR) posts cases and articles, hosts Tweet chats, advertises podcasts, and more on its Twitter account (@TheAJNR). The objective of this study was to determine whether user engagement is underestimated on the basis of publicly available metrics and to assess the engagement rate. This study demonstrated that engagement extends beyond visible metrics, suggesting an AJNR “silent” following beyond what is readily apparent. Median engagement rates from the @TheAJNR account from 2017 to 2019 appear stable since last reported in 2016 and are comparable with those reported in other professional medical journals.
ABBREVIATION:
- ER
- engagement rate
Twitter continues to be an active social media platform for radiologists, trainees, and academic organizations. Although Twitter was originally a personal communication platform, its dynamic and rapidly changing information stream encourages propagation of topics of interest in the medical community. Currently, this platform is routinely used to disseminate information for large medical societies, including the American Society of Neuroradiology.
Twitter has become a catalyst for real-life meet-ups by conference attendees and allows virtual participation for those unable to attend. In the field of neuroradiology, Radmanesh et al1 first described social media incorporation and use patterns by analyzing data from the 2014 Annual Meeting of the American Society of Neuroradiology (ASNR). When tweets from the Journal of the American College of Radiology targeted topics of member interest, Web site traffic increased 31%, and unique Web site visitors increased 20%.2 Studies by Koontz et al3 and Patel et al4 found that social media use among radiologists and radiology trainees ranged from 59% to 91%, with usage rates varying by generation. Furthermore, Patel et al noted that 60% of radiologists in their study used Twitter for professional purposes. These high rates of use demonstrate a new digital era in which information is being consumed on-line.
While many of the interactions are visible to the public (retweets, likes, and replies), some of the more important metrics for content engagement are not visible (impressions, engagements, engagement rate, URL clicks) to the public but can be accessed by the account owner (Fig 1). The American Journal of Neuroradiology (AJNR) Twitter account (@TheAJNR) has been in existence since 2013, now garnering more than 6400 followers and 7300 tweets at the time of publication. Each @TheAJNR posted case and published article is paired with a unique URL link to route users to the corresponding AJNR Web page.
Twitter Analytics. The free Twitter Analytics page provides data for the account and for each individual tweet from the account over designated time periods that can be viewed on-line or exported to Excel (Microsoft, Redmond, Washington). Only the account user has access to this page. The exported data include impressions, engagements, URL clicks, retweets, likes, and all other metrics collected for this project.
All social media activity is designed to increase engagement with users. However, the type of engagement important for an academic journal such as AJNR includes URL clicks, in addition to likes and retweets. One objective of this study was to assess whether case views and article readership resulting from Twitter posts (indicated by the number of URL clicks) exceed the number of visible engagements (particularly, retweets and likes), implying a “silent” Twitter following and educational outreach beyond what is readily apparent. Another objective of the study was to assess the engagement rate of the @TheAJNR account since the most recent reporting of AJNR engagement rate data from 2015 to 2016.5
MATERIALS AND METHODS
Data Source
Twitter Analytics data were obtained from the AJNR Twitter account for tweets published from January 2017 to June 2019. Available Twitter Analytics data include impressions, engagements, and engagement rate for each tweet.
“Impressions” refer to the number of times a tweet passes through user timelines or is included in a search.
“Engagements” are defined by the total number of times a user interacted with a tweet through a click for any purpose (retweet, reply, follow, like, URL link, hashtag, photo, or tweet expansion). The engagements of interest for our study were retweets, likes, and URL clicks.
“Engagement rate” (ER) is defined by total number of engagements divided by total number of impressions.
Likes and retweets can be seen by any Twitter user and were considered publicly visible engagements. The number of URL clicks data is only accessible by the owner of the Twitter account, thus considered a form of private engagement only visible to the account owner.
Tweet types were classified into 3 categories: cases, articles, and other. The “other” category included Tweet chats, blog posts, news announcements, advertisements for podcasts, upcoming meeting information, and deadline reminders.
Statistical Analysis
Summary statistics were obtained for URL clicks, retweets, likes, and replies for each of the 3 tweet types from January to December 2017. Both the difference between the number of URL clicks and the number of retweets and the difference between the number of URL clicks and the number of likes were computed for each tweet of each type (AJNR case, article, and other). Both difference response variables were analyzed separately using a 1-way analysis of variance with unequal variances to test for a difference among the 3 tweet-type population mean differences. Subsequently, for both the URL-click-retweet differences and the URL-click-like differences, case and article tweets were compared using a statistical contrast that used a t test. Also, subsequent to the 1-way analysis of variance, for each tweet type, a t test was performed to determine whether there was sufficient statistical evidence to suggest that the population mean number of URL clicks was greater than that of retweets, and a similar t test was performed for URL clicks and likes. To assess the level of engagement based on URL clicks, we obtained an estimate of the percentage of combined case and article tweets that had at least 25 URL clicks, and similar percentages were obtained for that same set of combined tweets that had at least 50, at least 75, and at least 100 URL clicks. These statistical analyses were conducted using SAS 9.4 (SAS Institute, Cary, North Carolina).
Using Twitter Analytics data available from January 2017 through June 2019, we calculated descriptive statistics for engagement rate, which included medians along with 25th and 75th percentiles by year and overall. These statistics were obtained using SPSS, Version 23 (IBM, Armonk, New York).
RESULTS
Nine hundred one tweets were analyzed, including 173 cases, 366 articles, and 362 others for the first objective of the study. There was evidence to suggest that at least 2 of the population mean differences among the 3 tweet types were significantly different for the URL click-retweet differences (P < .001) and the URL clicklike differences (P < .001).
Of particular interest was the comparison between tweets for AJNR articles and AJNR cases. For the difference between URL clicks and retweets, the mean difference was statistically significantly greater for cases than articles (50.1 versus 13.0, P < .001). The same was true for the difference between URL clicks and likes (49.8 versus 11.8, P < .001).
For all 3 tweet types, the mean number of URL clicks was significantly greater than the mean number of retweets (cases, 54.6 versus 4.6, P < .001; articles, 18.1 versus 5.0, P < .001; other, 7.3 versus 1.5, P < .001) and the mean number of likes (case, 54.6 versus 4.9, P < .001; articles, 18.1 versus 6.3, P < .001; other, 7.3 versus 3.0, P < .001) (Table 1).
Tweet metrics by typea
The second objective of the study entailing an assessment of the engagement rate of the @TheAJNR account used 2187 tweets (893 from 2017, 911 from 2018, and 383 from January to June 2019). The median ER for 2017 was 4.1% (25th percentile = 2.3%, 75th percentile =7.3%); for 2018 was 3.0% (1.7%, 5.2%), and was 3.5% (2.0%, 6.0%) for the first 6 months of 2019. The overall median ER was 3.5% (1.9%, 6.2%).
DISCUSSION
Free to users and available only to the owner of an account, the Twitter Analytics page offers data to the user that extend far beyond publicly visible metrics. In addition to likes, retweets, and replies that are available to the public, the account owner can evaluate trends in the engagement of his or her tweets based on other private metrics such as impressions, detail expands, URL clicks, and engagement rate.
Our URL click-retweet and URL click-like differences highlight this engagement of URL clicks visible only to the account owner that far outnumber the popular publicly visible metrics of retweets and likes. This feature is of particular importance because URL clicks are the best indicator of AJNR readership. A user who clicks the URL link is interested in reading more than the small snippet of information previewed in the Twitter post (limited to 280 characters and 4 pictures). Such a user is actively engaged in the AJNR content, but that interaction will leave no publicly visible footprint unless that same user likes, retweets, or comments.
When evaluating the frequency of URL clicks, we found that almost 40% of case and article tweets combined from the AJNR account get at least 25 URL clicks. A little over 4% reach at least 100 URL clicks, with the most popular case during our study receiving 205 URL clicks (Tables 1 and 2).
Frequency of engagement (URL clicks) with cases and articles
Each week, the AJNR posts a “Classic Case” submitted by readers (Fig 2). On May 7, 2018, a case of CLIPPERS received only 9 retweets and likes each but had 134 URL clicks, which channel the Twitter user directly to the AJNR site for full viewing.6 Thirty percent of the engagement for this particular tweet was through a link click. For each aforementioned case, AJNR posts a quiz where users can vote on a diagnosis. The corresponding case quiz for that week received 1033 votes, at least 42 of which came from Twitter but possibly more through direct funneling to the Web site, equaling about 4% of total quiz votes.
Link clicks far exceed publicly visible metrics. For this AJNR Classic Case top tweet, there were only 9 retweets and 9 likes (A, black arrow and box), 2 publicly visible metrics. B, Twitter Analytics revealed that the link was clicked 134 times (black arrow and box), a metric only visible to the @TheAJNR Twitter account user and indicating a large silent-but-active following.
One social media primer for radiologists states that “engagements track the deepest level of interaction with social media” and are “the most powerful metric, one not available with print media.”7 When evaluating article engagement, we found that a recent review article posted on May 16, 2017, had 25 likes and 15 retweets; however, the article link was clicked 85 times, taking viewers directly to the article abstract.8 Web site analytics show that this article has been accessed 1433 times and downloaded 773 times. Here, a full 62% of the tweet engagement was in the form of a link click. Similar to a previous study by Hoang et al, 9 this study supports that posting AJNR content on social media may expose viewers to the society’s content and funnel traffic to the Web site, increasing member and nonmember engagement with existing materials diligently curated by AJNR. Certainly, most of these accessions and downloads are independent of Twitter, however the fact remains that accessions through URL clicks are higher than what meets the eye visually.
Our review of the literature revealed a paucity of studies that serve as benchmarks for end-user engagement on social media in the scientific community, but the most commonly used metric has been ER. Leary et al10 examined the use of a social media campaign at the Resuscitation Science Symposium at the American Heart Association’s Scientific Sessions using Twitter Analytics data of 8 professional bloggers who live-tweeted during the sessions and used dedicated hashtags. The median ER for these bloggers was 2.4%. In another study analyzing the engagement data for the Journal of Vascular and Interventional Radiology (JVIR), Wadhwa et al11 showed that the median ER for @JVIRmedia steadily increased with time from 1.96% in 2014 to 4.47% in 2017. Another similar study by Wadhwa et al5 analyzing Twitter Analytics data for AJNR from August 2015 to July 2016 showed a median ER of 3.4%. When we expanded on this prior study on AJNR Twitter, further analysis showed that the median ER for @TheAJNR in the years 2017 and 2018 and the first 6 months of 2019 (January to June) was 4.1%, 3.0%, and 3.5%, respectively. The median ERs from our study are similar to the median ERs from other studies related to medical journals and use of social media described above. Further studies by other professional societies will be needed to more accurately define end-user engagement benchmarks for our industry, which include ER.
It is known from the prior work of Wadhwa et al5 that tweets containing an image or a hashtag or published in the morning hours have a higher ER. Their study also found that AJNR cases were the most popular style of tweet, concordant with the current study.5
There are several important limitations of this study. For instance, the presence of a URL click does not imply that a viewer is reading the article in full or looking through an entire case, and this has been reported previously by Hoang et al.9 Next, it cannot be ascertained what proportion of AJNR Web site activity is driven by Twitter at the current time, but this represents an area for future growth if the hope is to direct traffic to the AJNR Web site through social media. Last, the study was performed during a relatively short time frame since one of the authors (J.L.M.) assumed Twitter editorship in January 2017.
CONCLUSIONS
The AJNR Twitter account has an ever-present silent following with a far broader audience reach than previously recognized. This study suggests that the extent of viewership and on-line engagement should not be measured by likes and retweets of AJNR cases and articles alone. Rather, the number of URL clicks is more representative of the viewership of AJNR on-line content and outnumbers both retweets and likes. Also, median engagement rates from the @TheAJNR account from 2017 to 2019 appear stable since last reported in 2016 and are comparable with those reported in other professional medical journals, helping to inform the end-user engagement benchmarks for this industry.
Footnotes
Paper previously presented at: Annual Meeting of the American Society of Neuroradiology, June 2–7, 2018; Vancouver, British Columbia, Canada.
References
- Received July 31, 2019.
- Accepted after revision September 16, 2019.
- © 2019 by American Journal of Neuroradiology
