Nazli Goharian

Research in Information Retrieval and Mining

Jump to...

Domain Specific Search & Mining: Mining Social Media for Healthcare

Online discussions of virtually all topics are increasing; this phenomenon is ever more so in the domain of healthcare. Mining social media presents its difficulty due to the lay people / informal language used. Thus, it is a challenge to identify variations of the same concept and map to the expert term in the available knowledge sources. In this project we proposed and evaluated synonym discovery methods and concept extraction methods. One of the important applications is to harness these publicly available statements, to further our knowledge and understanding about drug behavior. We focus on using several drug related and other general social media sites, query analysis, peer-to-peer, and Web sites to detect expected and unexpected adverse reaction to drugs and devices. To understand users intentions, we utilize consumer medical terminology from UMLS and various other approaches to generate an adverse reaction synonym set that we use to identify both expected adverse reactions, as already recorded by the FDA, and unexpected adverse reactions mentioned in online reviews. ADRs Background (drug) language is utilized to evaluate the strength of the detected unexpected ADRs. Existing synonym discovery methods perform poorly when faced with the realistic task of identifying a target term's synonyms from among many candidates. We approach domain-specific synonym discovery as a graded relevance ranking problem in which a target term's synonym candidates are ranked by their quality. In this scenario a human editor uses each ranked list of synonym candidates to build a domain-specific thesaurus. We evaluate our method for graded relevance ranking of synonym candidates and find that it outperforms existing methods. We reduce the impact of incomplete information by learning the relationship between user mentions of symptom,conditions and drugs. Furthermore, we utilize our synonym discovery and concept extraction methods to construct a framework to detect trends. We furthermore studied the effects of twitter sampling on the trend detection, as well as methods to quantify the influence of news cycle on social media activity.

  • A.Yates, A. Kolcz, N. Goharian and O. Frieder, "Effects of Sampling on Twitter Trend Detection", In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC’16), May 2016.
  • A. Yates, J. Joselow, N. Goharian, "The news cycle's influence on social media activity", International AAAI Conference on Web and Social Media (ICWSM). May 2016.
  • A. Yates, N. Goharian, O. Frieder, Learning the relationships between drug, symptom, and medical condition mentions in social media", International AAAI Conference on Web and Social Media (ICWSM). May 2016.
  • J. Parker, A. Yates, N. Goharian, and O. Frieder, "Health Related Hypothesis Generation using Social Media Data," Journal of Social Network Analysis and Mining, Springer, 2015.
  • A. Yates, N. Goharian, O. Frieder, "Extracting Adverse Drug Reactions from Social Media", Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI-15), 2015.
  • A. Yates, J. Parker, N. Goharian, and O. Frieder, A Framework for Public Health Surveillance, In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC.14), May 2014.
  • A. Yates, N. Goharian, O. Frieder, Relevance-Ranked Domain-Specific Synonym Discovery, in 36th European Conference on Information Retrieval (ECIR), April 2014 (to appear).
  • J. Parker, Y. Wei, A. Yates, O. Frieder, N. Goharian, A Framework for Detecting Public Health Trends with Twitter., The 2013 IEEE/ACM International Conference on Advances in Social Network Analysis and Mining, Aug. 2013.
  • A. Yates, N. Goharian, O. Frieder, .Extracting Adverse Drug Reactions from Forum Posts and Linking them to Drugs., SIGIR Workshop on Health Search and Discovery, July-Aug 2013.
  • A. Yates, N. Goharian, and O. Frieder, Graded Relevance Ranking for Synonym Discovery, 22nd International Conference on World Wide Web (WWW), May 2013 (short).
  • A. Yates and N. Goharian, ADRTrace:Detecting Expected and Unexpected Adverse Drug Reactions from User Reviews on Social Media Sites, in 35th European Conference on Information Retrieval (ECIR), 2013. (short)
  • A. Yates, N. Goharian, Mining Social Media for Healthcare, ICBI Biomedical Informatics Symposium at Georgetown University, Best Poster Award (1 out of 28), Oct 2012.

Scientific Summarization

Due to the expanding rate at which articles are being published in various scientific fields, it has become difficult for researchers to keep up with the new developments. Scientific summarization aims to facilitate this problem. One useful strategy for scientific summarization is citation based summarization in which citations to a reference article are used to generate the summary of the reference paper. While citations have been previously used in generating scientific summaries, they lack the related context from the referenced article and therefore do not accurately reflect the article’s content. Our goal is to overcome this problem by providing the appropriate context for the citations and utilize this information towards extractive summary of the article. We have also shown that using scientific article’s inherent discourse structure can help improving the quality of the generated summaries. We are currently investigating approaches for development of more robust general summarization and scientific summarization routines.

  • A. Cohan and N. Goharian, "Revisiting Summarization Evaluation for Scientific Articles", In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC’16), May 2016.
  • A. Cohan and N. Goharian, "Scientific Article Summarization Using Citation-Context and Article's Discourse Structure" Empirical Methods for Natural Language Processing (EMNLP) 2015.
  • A. Cohan, L. Soldaini, and N. Goharian, "Matching Citation Text and Cited Spans in Biomedical Literature: a Search-Oriented Approach," NAACL-HLT, 2015.
  • A. Cohan, L. Soldaini, S. Mengle, and N. Goharian, "Towards Citation-Based Summarization of Biomedical Literature," Text Analysis Conference (TAC), 2014.

Clinical Decision Support Systems

Keeping current given the vast volume of medical literature published yearly poses a serious challenge for medical professionals. Thus, interest in systems that aid physicians in making clinical decisions is intensifying. We explore and evaluate approaches to retrieve relevant medical literature given a medical case report. Furthermore, given the action a health expert is seeking to complete (make a diagnosis, prescribe a treatment, or order a test), we investigate reranking techniques that could provide more appropriate literature.

  • L. Soldaini and N. Goharian, "QuickUMLS: a fast, unsupervised approach for medical concept extraction", In Proceedings of the Medical Information Retrieval (MedIR) workshop at SIGIR 2016, July 2016.
  • L. Soldaini, A. Cohan, A. Yates, N. Goharian, O. Frieder, "Retrieving Medical Literature for Clinical Decision Support", Proceedings of the 37th European Conference on Information Retrieval (ECIR 2015), 2015.
  • L. Soldaini, A. Cohan, A. Yates, N. Goharian, O. Frieder, "Query Reformulation for Clinical Decision Support Search", Proceedings of the 23rd Text REtrieval Conference Proceedings (TREC), 2015.
  • A. Cohan, L. Soldaini, A. Yates, N. Goharian, and O. Frieder, "On clinical decision support", Proceedings of the 5th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics, 2014. (short)

Opinion Mining and Sentiment Analysis of User Reviews; Joint collaboration with Orbitz Worldwide

User reviews are commonly used by both the Web users and the providers of goods and services on the Web. Thus, analyzing and understanding users. reviews plays a pivotal role for the decision making process of both parties. As the popularity of online user reviews continues to increase, it is becoming increasingly difficult for potential customers and even business owners to understand what aspects business reviewers cared about and how the reviewers felt about those aspects. Many websites allow and even encourage people to submit reviews of various products and services. The text within these reviews often contains valuable information not found in a single 1-5 "star rating". This research proposed and evaluated a novel approach to efficiently model and analyze the text within user reviews to estimate how much reviewers care about different aspects of a product (i.e., amenities, food, location, room, etc. of a hotel) by estimating the aspects' weights. A vector of aspect weights synthesizes the average customer's preferences and expectations as well as the product's actual performance, thus providing a way to characterize the subject of the reviews. This approach performs statistically similar to, and arguably better than, the best existing method, but with significantly lower computational complexity (linear time). While the current domain of this research is a hotel review data set, this method is not domain-specific and should work for other types of reviews.

Query Session Analysis

We developed and evaluated our approach that utilized our earlier research on identifying the relationships among topics, now to understand the topic of user queries and intent given sequence of user queries from a session or sessions. The context of the session queries is utilized to improve the effectiveness of identifying the intent or topic of current query. Earlier efforts utilized fixed number of preceding queries to derive such contextual information. We proposed and evaluated an approach (DQW) that identifies a set of "unambiguous" preceding queries in a dynamically determined window to utilize in classifying an ambiguous query to a topic. Furthermore, utilizing a relationship-net (R-net) that represents relationships among known topics, we improved the classification effectiveness for those ambiguous queries whose predicted topic in this relationship-net is related to the topic of a query within the window. Our results indicated that the hybrid approach (DQW+R-net) statistically significantly improves the Conditional Random Field (CRF) query classification approach when static query windowing and hierarchical taxonomy are used (SQW+Tax), in terms of precision (10.8%), recall (13.2%), and F1 measure (11.9%). The findings of this research can improve our understanding of user query intent and consequently the search results.

Personalized Ranking Twitter Friends: Who to follow or not to follow

One of the challenges for the users of social media, such as in Twitter, is the fast growing number of people each user is following. The features available in Twitter provide meaningful information that can be harvested to provide a ranked list of "friends" (i.e., followees) to each user. We hypothesize that retweet and mention features can be further enriched by incorporating both temporal and additional/indirect links from within user's community.

  • Y. Zhu and N. Goharian, To Follow or Not to Follow: A Feature Evaluation., 22nd International Conference on World Wide Web (WWW), May 2013 (short).

Finding Category / Topic Relationships

The hierarchical nature of existing Web directories, ontologies, and folksonomies, are known to provide meaningful information that guide users and applications. Knowledge of relationships among text categories is of interest in different domains such as text classification, content analysis, text mining, query [session] understanding. Knowledge of relationships among categories is of the interest in different domains such as text classification, content analysis, and text mining. We propose and evaluate approaches to effectively identify relationships among document categories. Our proposed novel method capitalizes on the misclassification results of a text classifier to identify potential relationships among categories. This leads to a relationship network. We demonstrate that our system detects such relationships, even those relationships that assessors failed to identify in manual evaluation. Furthermore, we favorably compare the effectiveness of our methods with the state of art method and demonstrate a significant improvement in precision and recall. Furthermore, we are interested to discover interesting relationships in the existing hierarchical knowledge representations. The hierarchical nature of existing Web directories, ontologies, and folksonomies, are known to provide meaningful information that guide users and applications. We hypothesize that such hierarchical structures provide richer information if they are further enriched by incorporating additional links besides parents, and siblings, namely, between non-sibling nodes. We call such structure a "networked hierarchy". Our empirical results indicate that such a networked hierarchy introduces interesting links between nodes (non-sibling) that otherwise in a hierarchical structure are not evident. This research findings can be utilized to improve and maintain the existing hierarchies, construct topic hierarchies or networks, and improve our understanding of topic hierarchies in text search and query session research.

Passage Detection

Passages can be hidden within a text to circumvent their disallowed transfer. Such release of compartmentalized information is of concern to all corporate and governmental organization. We explore the methodology to detect such hidden passages within a document. A document is divided into passages using various document splitting techniques, and a text classifier is used to categorize such passages. We present a novel document splitting technique called dynamic windowing, which significantly improves precision, recall and F1 measure.

Feature Selection in Text Classification

With the ever-increasing number of documents on the web, digital libraries, news sources, etc., the need of a text classifier that can classify massive amount of data is becoming more critical and difficult. The major problem in text classification is the high dimensionality of feature space. The Support Vector Machine (SVM) classifier is shown to perform consistently better than other text classification algorithms. However, the time taken for training a SVM model is more than other algorithms. We explore the use of the Ambiguity Measure (AM) feature selection method that uses only the most unambiguous keywords to predict the category of a document. Our analysis shows that AM reduces the training time by more than 50% than the scenario when no feature selection is used, while maintaining the accuracy of the text classifier equivalent to or better than using the whole feature set. We empirically show the effectiveness of our approach in outperforming seven different feature selection methods using two standard benchmark datasets

Misuse (Off-Topic Search) Detection in Information Retrieval

Most computer crime traditionally has been the "insider" problem. In fact after virus, i.e., malicious code, insider abuse, called misuse, is the second most threatening attack. We focus the problem on misuse of search systems. Misuse detection is an attack to the system by an authorized user who is misusing their privileges. Prior work on misuse detection mainly focused on using logs and user profiles. Profile-based detection systems audit the deviation of user activities from normal user profiles. A user's command history is reviewed based on the percentage of commands used over a specific period of time and logs are mined. We developed algorithms and implemented a misuse detection system by comparing user behavior to user interest profile learned through clustering, relevance feedback, and finally fusion of results of these methods. We evaluated our system by setting up both an automatic and manual (four human evaluators) evaluation systems and showed a significant improvement in detection rate.

Spam Detection

Although different than misuse detection and passage detection, yet has similar goal of detecting something "bad" and"wrong". Spam detection has historically focused on email spam. However, with ever increasing sources of short texts, on the order of 10s of characters, such as in twitter and mobile phone texting, it is important to be able to detect spam where the text provides such little information. We examined the affect of various text-based features such as various character-grams, word grams, length, and specific words such as .rate., .award., etc. to classify spam. We found that simple textual features such as n-character grams are good indicators. Our system showed improvement over the state of the art. I am continuing my research in this area to improve the detection rate and potentially to apply to different type of short text

Text Extraction; Joint collaboration with America Online Inc. (AOL)

Significant portion of the data on the World Wide Web is in the form of HTML pages. Since content, navigational information, advertisement, and formatting have no clear separation in HTML, the conventional information retrieval systems have the additional task of dealing with noisy data when providing full-text search. A problem that is not well studied is the negative effect of such noise data on the result of the user queries. Removing these data improves the effectiveness of search by reducing the irrelevant results. Furthermore, we argue that the irrelevant results, even covering a small fraction of retrieved results, have the restaurant-effect, namely users are less likely to return or use the search service after a bad experience. This is of more importance, considering the fact that an average of 26.8% of each page is formatting data and advertisement. We developed an algorithm and implemented the system. Our experimental results demonstrated that using extraction reduces the irrelevant results for the queries that generate "bad" results. Our experimental results shows that if one would use a system with extracted text instead of non-extracted text, then a significant improvement can be achieved on irrelevant results retrieved by the engine based on non-extracted text. On our experiment with cnnfn collection and AOL user queries, we improved all bad results.

Medical Informatics

One of the areas in Medical Informatics concerns searching biomedical literature, which differs from conventional search in that the vocabulary (terms) involve significantly different grammatical structures. Suffixes vary in nature; synonyms are far more common; the reliance of taxonomies is far greater. This research addresses issues raised by this domain. Another area is Data collection for clinical research, which is quite fragmented in the field of medicine. Issues of administrative hurdles associated with multi-center studies, error free data collection, automated analysis, and increased collaboration among different medical research centers is of concern. The database and data mining techniques allow for error free and automated analysis. In collaboration with the Northwestern Medical School, we designed and developed a computer-assisted medical application that captured data needed to study the effectiveness of the diagnosis and treatment of Urinary Tract Infections (UTIs). Furthermore, we develpped a data collection and analysis system, via which the application of LithoTron® lithotripter on the patients with kidney stone were analyzed. Yet another area of my recent interest in Medical Informatics relates to mining social media for finding patterns and opinions of patients on various treatments of specific dieseases.