Carnegie Mellon University


Carnegie Mellon’s Software and Societal Systems Department (S3D) hosts an active research group with a highly interdisciplinary approach to software engineering. Indeed, we believe that interdisciplinary work is inherent to software engineering. The field of software engineering (SE) is built on computer science fundamentals, drawing from areas such as algorithms, programming languages, compilers, and machine learning. At the same time, SE is an engineering discipline: both the practice of SE and SE research problems revolve around technical solutions that successfully resolve conflicting constraints. As such, trade-offs between costs and benefits are an integral part of evaluating the effectiveness of methods and tools.

Emerging problems in the area of privacy, security, and mobility motivate many challenges faced by today’s software engineers, motivating new solutions in SE research. Because software is built by people, SE is also a human discipline, and so research in the field also draws on psychology and other social sciences. Carnegie Mellon faculty bring expertise from all of these disciplines to bear on their research, and we emphasize this interdisciplinary approach in our REU Site. Below, you'll find projects we are planning for Summer 2022.

Mentors: Jim Herbsleb and Laura Dabbish

Description and Significance
How do open source contributors think about AI Ethics? Critical AI Ethics research and discourse often focuses on issues in proprietary AI made by private tech companies, but open source is different: radical code transparency allows anyone to inspect for ethics issues, but also allows the proliferation of harmful or easily misused code. Organizational structures and incentives are often very different in open source, and many contribute anonymously, complicating accountability. Therefore, challenges that professionals doing AI Ethics work in a tech company setting are likely to look very different in an Open Source context. But, because powerful AI systems are developed in this open source context, it is important to understand AI Ethics issues from the perspective of open source contributors: how, when, and why do contributors raise Ethics issues? How do they mitigate them? What do they feel is in or out of their control?

Student Involvement
There are a variety of opportunities for students to contribute depending on their interest and background. We could use NLP techniques to quantitatively examine where ethics issues are likely to be raised in GitHub issue threads, project ReadMes, commit logs, licenses, or contributor bios, using any of SQL, Python, R, topic modeling or sentiment analysis. In a different direction, we could survey and interview contributors to qualitatively examine what ethics issues may arise in their project, and understand what power they feel they have to mitigate them. We could also build code analysis tools to help open source communities understand how their project is used downstream in perhaps ethically dubious ways. The ideal student has an interest in empirical methods, is excited to read literature across disciplines, and is motivated to help AI Ethics discourse encompass an expanding set of issues and AI communities.


Metcalf, Jacob, and Emanuel Moss. "Owning ethics: Corporate logics, silicon valley, and the institutionalization of ethics." Social Research: An International Quarterly 86.2 (2019): 449-476.

Orr, Will, and Jenny L. Davis. "Attributions of ethical responsibility by Artificial Intelligence practitioners." Information, Communication & Society 23.5 (2020): 719-735.

Dabbish, L., Colleen Stuart, Jason Tsay, Jim Herbsleb. "Social coding in GitHub: transparency and collaboration in an open software repository." Proceedings of the ACM 2012 conference on computer supported cooperative work. (2012): 1277-1286.

Mentor: Brad Myers

Description and Significance
When learning a new skill or completing a complex task, learners amass a large set of contextualized knowledge that could benefit later users. However, the process of writing down and synthesizing that knowledge into something that could be understood by others in the future is a time-consuming process that many learners are not interested in doing. We hypothesize that supporting short notes through in-context annotations will make the process of jotting down information and sharing this information with others easier while benefiting the original author through providing features to help learners organize their information and manage their tasks. We have had success with our annotation tool, Adamite (see, a Chrome extension that helps developers share information when using programming documentation, but we are interested in generalizing our approach to other kinds of content, including integrating with other platforms such as Google Docs, Gmail or Notion where people make comments, supporting operating system-level notes such as notes on folders or files, and supporting code annotations in the IDE (we currently have an in-development VS Code extension). Participating in this research project may result in being an author on a publication at a prestigious conference such as CHI, ICSE, UIST, or CSCW, and we are hoping to release Adamite as an open source project for general use.

Student Involvement
Students working on this project will be expected to have significant web development and/or JavaScript experience (e.g., at least 1 completed course in web development). We would also like the student to have experience using React and/or TypeScript.

Read our pre-print here:

Mentors: Vincent Hellendoorn, Brad Myers, and Bogdan Vasilescu

Description and Significance
To fully understand and use an API properly, a developer needs to know many things, from domain-specific concepts (e.g., how reinforcement learning works) and design patterns (e.g., MVC pattern), to details about each API (e.g., how to use a specific API method). However, official API documentation, which is known to be the primary source of information for developers [1], is often incomplete or has poor usability [2]. Thus, programmers must rely on external sources (e.g., Stack Overflow) to compensate for the shortcomings of the official documentation [2].

To help developers access useful information more conveniently, we are building an API knowledge base. We have developed approaches to automatically extract API-relevant information from source repositories [3] and Stack Overflow. In this project, we want to investigate algorithms to automatically organize the collected information, so developers can more easily retrieve necessary information for a variety of purposes.

Student Involvement
In this project, an REU student will contribute to the design and implementation of a new API knowledge base. The REU student could work on developing or improving mining approaches to gather knowledge from various sources. To extract useful knowledge from the unstructured data (e.g., code, natural language description), the student may use machine learning techniques, such as Transformer language models [4]. Or the REU student could help design and implement a knowledge graph to efficiently organize the collected API information. The student could also help train a machine learning model for the API information retrieval.

The exact project will depend on the student’s interest as well as project needs at the time. The ideal student has an interest in information extraction, knowledge graph building, and/or API usability. Previous knowledge in machine learning / natural language processing is a plus.

[1] T. C. Lethbridge, J. Singer, and A. Forward. 2003. How software engineers use documentation: the state of the practice. IEEE Software 20, 6: 35–39.

[2] Emad Aghajani, Csaba Nagy, Olga Lucero Vega-Márquez, Mario Linares-Vásquez, Laura Moreno, Gabriele Bavota, and Michele Lanza. 2019. Software Documentation Issues Unveiled. In Proceedings of the 41st International Conference on Software Engineering (ICSE ’19).

[3] Daye Nam, Amber Horvath, Andrew Macvean, Brad Myers, Bogdan Vasilescu. 2019. MARBLE: Mining for Boilerplate Code to Identify API Usability Problems. In Proceedings of the 34th International Conference on Automated Software Engineering (ASE ’19)

[4] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. "Attention is all you need." In Advances in Neural Information Processing Systems (NIPS ’17).

Mentors: Rashmi Vinayak and Greg Ganger

Description and Significance
We are designing the next generation of distributed systems for large-scale data storage, such as those used for youtube and cloud storage. By combining on-line analysis of component behaviors with automatic reconfiguration, our research will enable distributed storage to be less expensive, be more reliable, and have reduced climate footprint.

The big idea behind this research project is that we need to change the distributed system design and policies to recognize and exploit the differences among their component storage devices—they aren’t all the same and very much should not be treated like they are. Large data centers include millions of storage devices---a mix of Flash-based solid-state disks (SSD), mechanical hard disks (HDD) and (very recently) non-volatile memory (NVM). While HDDs are treated differently than SSDs, the reliability and throughput:capacity differences among devices of a given technology are generally ignored… which our research has exposed as creating very wasteful (in terms of performance, number of devices required, and energy consumption). Our research aims at exploiting the various dimensions of heterogeneity to build automatically-heterogeneity-aware cluster storage systems that can better utilize the underlying devices. For example, based on real-world deployment logs at companies like Google, we have shown large reductions in capacity waste by adapting data redundancy schemes to device-specific robustness levels. We are also finding significant opportunities in changing data placement schemes to address device-capacity heterogeneity to greatly improve efficiency.

Student Involvement
Students can get firsthand experience with any of several aspects of this research, depending on best fit for interests and skills. There are many avenues for contribution, including software prototyping of new policies/schemes, detailed experimentation with real systems, and data analyses of and ML model creation from real system logs. This research project is devising and evaluating new storage system designs to automatically identify and exploit storage device heterogeneity. Doing so involves a mix of processing of real system logs (device deployment, failures, and workloads), simulations driven by those logs, and system prototyping and experimentation via integration of heterogeneity-aware policies into a real distributed storage system, such as HDFS.

[1] PACEMAKER: Avoiding HeART Attacks in Storage Clusters with Disk-adaptive Redundancy. Saurabh Kadekodi, Francisco Maturana, Suhas Jayaram Subramanya, Juncheng Yang, K. V. Rashmi, Gregory R. Ganger. 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI'20), Nov. 4–6, 2020.

[2} Cluster Storage Systems Gotta Have HeART: Improving Storage Efficiency by Exploiting Disk-reliability Heterogeneity. Saurabh Kadekodi, K. V. Rashmi, Gregory R. Ganger. 17th USENIX Conference on File and Storage Technologies (FAST '19) Feb. 25–28, 2019.

Mentors: Lori Levin, David Mortensen, and Shinji Watanabe

Description and Significance
This project combines work of indigenous language communities, linguists, and machine learning experts. We plan to build end-to-end neural networks that assist with language documentation. The input will be audio recordings and the output will be a transcription where the words are segmented into prefixes, stems, and suffixes, and are annotated with grammatical labels.

There are about 7000 human languages currently spoken, more than half of which are in dire need of documentation. Language documentation is the process of collecting and preparing data for rigorous study by linguists or archiving for cultural preservation. Documentation includes recording, transcribing, translating, adding grammatical labels, and adding metadata. Without automation, transcription of language recordings takes about ten times real time. Recent advances in ASR (Automatic Speech Recognition) and NLP (Natural Language Processing) have enabled partial automation of the documentation process, greatly increasing the amount of data that can be processed for each language. Speakers of indigenous languages who have graduate degrees in linguistics and other subjects will be available as mentors. Some languages we may focus on are Mixtec, Nahuatl, and Amuzgo from Mexico and Inupiaq from Alaska. However, speakers of any indigenous language are welcome to work on their own language.

Student Involvement
Students from different disciplines will be involved in different parts of the project including ASR, NLP, and neural networks. In addition, students who speak indigenous languages or have a background in linguistics can be involved in labeling data, resolving linguistic issues in how to label data, and conducting error analysis.


Automatic Interlinear Glossing for Under-Resourced Languages Leveraging Translations, 2020, Xingyuan Zhao, Satoru Ozaki, Antonios Anastasopoulos, Graham Neubig, Lori Levin, Proceedings of the 28th International Conference on Computational Linguistics (COLING).

Large-Scale End-to-End Multilingual Speech Recognition and Language Identification with Multi-Task Learning, 2020, W Hou, Y Dong, B Zhuang, L Yang, J Shi, T Shinozaki
Proceedings of the Interspeech.

Leveraging End-to-End ASR for Endangered Language Documentation: An Empirical Study on Yolox\'ochitl Mixtec, 2021, Jiatong Shi, Jonathan D. Amith, Rey Castillo García, Esteban Guadalupe Sierra, Kevin Duh, Shinji Watanabe, Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (ACL).

Mentor: Yuvraj Agarwal

Description and Significance
We are increasingly surrounded by IoT devices that sense a wide variety of information in their vicinity. Our lab has developed the "" platform (check out our website) that allows sensing of a wide variety of physical phenomenon such as light, sound, vibrations, audio and wireless signals broadcast from devices in their vicinity. Furthermore, our machine learning stack converts these sensed values into intelligent inferences like "a phone rang" or "door open/closed" etc. We have deployed this infrastructure in an actual CMU campus building. Given the wide variety of sensed phenomenon, our goal is to study the privacy implications of this sensing infrastructure and devise new privacy abstractions, novel user interfaces to notify building occupants about sensors, privacy preserving data-analysis algorithms, mobile Apps to give users control on what they share with whom, etc. This will be the largest IoT infrastructure and testbed of its kind.

Student Involvement
The student involved will work closely with two faculty member and one or more of the PhD students on this project. There are numerous aspects of this project that the selected student can work on. For example, you could work on building the privacy primitives in our underlying Building operating system (GioTTO), including an App Service for 3rd party Apps on our platform. You could also work on implementing new privacy aware APIs that app developers will use to develop apps around the Mites data. Depending on the students interest and expertise, there are also projects around conducting user studies around Apps built on this infrastructure and accompanying Mobile Apps. We are looking for candidates who have strong CS or EE fundamentals and are strong at programming. Also, it would be a plus to have some experience in building systems and/or Mobile Apps. Most importantly, the candidate should be interested in building real systems and care about data privacy!

[1] "Ask The Experts: What Should Be On An IoT Privacy And Security Label?" Pardis Emami-Naeini, Yuvraj Agarwal, Lorrie Faith Cranor, Hanan Hibshi. Oakland S&P 2020 -- The 41st IEEE Symposium on Security and Privacy, 2020.

[2] "PrivacyStreams: Enabling Transparency in Personal Data Processing for Mobile Apps." Yuanchun Li, Fanglin Chen, Toby Jia-Jun Li, Yao Guo, Gang Huang, Matthew Fredrickson, Yuvraj Agarwal, Jason I. Hong. Ubicomp 2017 -- ACM Transactions on Interactive, Mobile, Wearable Ubiquitous Technology (IMWUT), September, 2017.

[3]"Toward Building a Safe, Secure, and Easy-to-Use Internet of Things Infrastructure." Yuvraj Agarwal, Anind Dey. IEEE Computer 2016: IEEE Computer Society.

[4] CMU GIOTTO Stack:

[5] SynergyLabs Mites Stack:

Mentor: Eunsuk Kang

Description and significance
Given that network protocols are ubiquitous in critical infrastructures, any vulnerability or flaw in widely used protocols could cause damage to our economy, environment, and even human lives. The challenge is that these protocols are deployed in evolving uncertain and malicious environments, which can compromise important protocol properties such as security and reliability. Therefore, these network protocols must be robust against these uncertain environments. The current robustness analysis of network protocols only considers specific network faults, e.g., packet duplication and bit corruption, and specific attack models, e.g., the Dolev-Yao attack model. However, it is unclear how the protocol will behave outside these specific scenarios. In this project, we are developing automated reasoning techniques for evaluating a network protocol with respect to its robustness against an adversarial environment; i.e., the degree of assumptions about network environment capabilities under which the protocol is capable of maintaining its properties.

Student involvement
In this project, students will work on methods for specifying, analyzing, and testing network and security protocols. Potential research tasks include: (1) develop formal models of network protocols; (2) analyze the protocol robustness with respect to various security and reliability requirements; (3) test the robustness of protocol implementations. Students are expected to have a basic background in logic and discrete mathematics, and will learn about the-state-of-the-art techniques in formal methods, software analysis, testing, and programming languages.

Mentors: Bradley Schmerl and David Garlan

Description and significance
Self-adaptive systems (e.g., robots or vehicles) need to plan their actions under the consideration of multiple quality attributes. For instance, energy consumption, cost, performance, and safety need to be balanced and traded off against each other. In previous research, we developed approaches to explain tradeoffs in adaptation plans that are based on machine learning techniques (e.g., Principal Component Analysis and Decision Tree Learning) [1,2]. The current approach relies on graphical plots that can help experts to understand how different quality attributes are related to each other and how they impact the generated planning actions. However, these plots are difficult to understand for end users. The goal of this project is to build usable tool support for quality tradeoff explanations, with a focus on a user interface that provides understandable explanations to human users.

Student involvement
Students will be involved in designing specific solutions for parts of the problem, for example, in eliciting requirements for a visual tradeoff explanation tool, designing a graphical user interface and conceptual solutions to visualize quality tradeoffs, and creating a prototype application for a suitable example self-adaptive system. The ideal student has an interest in self-adaptive systems and user interface design. Previous knowledge in machine learning is a plus.

[1] Cámara, J., Silva, M., Garlan, D., & Schmerl, B. (2021, September). Explaining Architectural Design Tradeoff Spaces: A Machine Learning Approach. In European Conference on Software Architecture (pp. 49-65). Springer, Cham.

[2] Wohlrab, R., Cámara, J., Garlan, D. & Schmerl, B. Explaining Quality Attribute Tradeoffs in Automated Planning for Self-Adaptive Systems. In submission.

Mentor: Hanan Hibshi

Description and Significance
Most software engineers come from a computer science/computer engineering background. Secure coding practices, software bugs and vulnerabilities are usually taught as a separate course/unit in our CS/CE curriculum. One could argue that this practice could not be in-line with the security-by-design advice that we continue to give developers and could result in developers treating security as a secondary task. How would we expect a developer to incorporate security by design if they haven’t been trained that way from the beginning?

In this research, we will survey traditional programing and coding curriculum and we will evaluate example code snippets used in these courses against known secure coding practices. Based on the analysis, we look to further provide a set of recommendations for educators and instructors who teach CS/CE courses to incorporate secure coding practices in their curriculum from the start so we can help achieve with the security-by-design goals.

Student Involvement
Students will be surveying the literature and examining different examples used in teaching materials for introductory and mid-level programming courses. Students will learn how to conduct a systematic literature review and how to synthesizes large amount of information, draw conclusions, and present their findings. Throughout the process, students will be learning about best secure programming practices.

Mentor: Amy Ogan

Description and Significance
We are envisioning the classroom of the future by combining and leveraging cutting edge technologies of today. Our efforts are motivated by the fact that student attention, engagement, and confusion, and even teachers' own behaviors are hard to attend to in real time, and teachers lack the quantitative data that would help them transform their classroom experience to improve learning outcomes.

We have developed a novel classroom sensing system that captures rich student/instructor metrics, and classroom context non-intrusively through cameras in the classroom. We have collected data over 400+ classroom sessions across 30+ courses.

Currently, we are working towards extending our research in multiple directions. For example, we seek to understand the challenges and opportunities of collecting classroom data for various relevant stakeholders (e.g. instructors, students, administrators, researchers, etc.) and build interfaces for stakeholders to derive value from our system (e.g. dashboards for instructors to reflect on their sessions, data for students to make more informed decision during their course selections, or visualizations for building management to design better classrooms etc.). There are numerous design and research challenges in these broad directions, and we welcome any undergraduates who have an interest in design, learning technologies, and user research.

Student Involvement
The student will be involved in conducting user studies and data analyses towards building classroom analytics systems for relevant classroom stakeholders. Desired skills are prototyping in tools like Figma, front-end web development in HTML/CSS and JavaScript, experience conducting interviews and user studies.

Ahuja, K., Kim, D., Xhakaj, F., Varga, V., Xie, A., Zhang, S., ... & Agarwal, Y. (2019). EduSense: Practical classroom sensing at Scale. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(3), 1-26.

Ahuja, K., Shah, D., Pareddy, S., Xhakaj, F., Ogan, A., Agarwal, Y., & Harrison, C. (2021, May). Classroom Digital Twins with Instrumentation-Free Gaze Tracking. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-9).

Mentors: Yuvraj Agarwal and Chris Harrison

Description and Significance
We are envisioning the classroom of the future by combining and leveraging cutting edge technologies of today. Our efforts are motivated by the fact that student attention, engagement, and confusion, and even teachers' own behaviors are hard to attend to in real time, and teachers lack the quantitative data that would help them transform their classroom experience to improve learning outcomes.

We have developed a novel classroom sensing system that captures rich student/instructor metrics, and classroom context non-intrusively primarily through computer vision and machine learning pipeline. In seeking a lightweight, minimally-invasive sensing solution, we have built our current prototype system around IP cameras in the classrooms and couple that with a distributed backend processing system. We have collected data over 400+ classroom sessions across 30+ courses.

Currently, we are working towards extending our base sensing system in multiple directions. We are building and analytics pipeline, which can capture broader, human interpretable context from the low level inferences, and creating multiple applications on the top of it to allow relevant stakeholders to derive value from our system, i.e allowing instructors to reflect on their sessions, or allowing students to make more informed decision during their course selections, or allowing building management to design better classrooms etc. We are also working towards improving our underlying sensing system to incorporate richer low level features from raw audio/video data.

There are numerous research challenges in these broad directions, and we welcome any undergraduates who loves building systems or doing large scale analytics, and are interesting in joining hands with us to bring some of these ideas to reality. Any other relevant ideas which can utilize our sensing infrastructure to contribute towards improving classroom experiences are also welcome.

Mentor: Rohan Padhye

Description & Significance
Fuzz testing is a popular technique for finding software bugs and security vulnerabilities using randomized test-input generation. State-of-the-art fuzzing tools perform coverage-guided fuzzing: they evolve a corpus of inputs that maximize code coverage (e.g. lines of code executed) in the program under test. However, modern software is continuously evolving. Software security therefore requires continuously fuzzing a moving target. It is impractical to run full fuzzing sessions after every code change—conventional fuzzing campaigns require days or weeks to complete and even incremental fuzzing [1] sessions require several hours to analyze the effect of code changes. As software complexity grows in terms of size and commit frequency, it becomes vastly more efficient to simply run a fixed suite of regression tests or to use such a suite as a seed corpus for a very short fuzzing session. Unlike the majority of fuzzing tools that look for bugs in a possibly incorrect program, this project targets fuzzing of an "assumed-to-be-correct" reference program with the goal of increasing future-fault detection ability by synthesizing a high quality regression test corpus.

Student Involvement
The project will involve the design and/or evaluation of highly efficient program analysis algorithms, software development in Java or C, analyzing real-world software (e.g. Apache Maven, Google Closure, OpenSSL), running experiments in the cloud, and perhaps even finding cool new bugs! As an example, the student might extend the recently developed Bonsai Fuzzing technique [1] for generating inputs that are not only concise by construction, but also execute efficiently and can be easily prioritized. The project will expose students to working with the internals of production-scale fuzzing tools such as JQF [2], algorithm design, experimental evaluation techniques, and on testing real-world software applications with the goal of improving software quality. Interested students should be comfortable or interested in hacking on non-trivial software systems. Some background with program representations (e.g. ASTs, JVM bytecode, or LLVM IR) will be helpful but is not required.

[1] V. Vikram, R. Padhye, and K. Sen. "Growing a Test Corpus with Bonsai Fuzzing". ICSE'21
[2] R. Padhye, C. Lemieux, and K. Sen. "JQF: Coverage-Guided Property-Based Testing in Java". ISSTA’19 —

Mentors: Jonathan Aldrich and Joshua Sunshine

Description and significance
Software verification is the process of ensuring that a software implementation does what it is intended to do (i.e., ensuring the software implementation adheres to its specification). Software verification is important for all software systems, but particularly so for critical applications such as control systems for aircraft and nuclear power plants. In particular, static verification involves writing detailed specifications (often in a formal logic) on system components, such as pre- and postconditions for functions. However, this approach does not support incrementality. To prove that a function adheres to its postcondition (e.g. that a findMax function returns the maximal element of a list), tools demand many additional specifications. Worse, even, is that a single specification cannot be checked for correctness without providing all other required specifications.

In 2018, we introduced gradual verification [1], which supports the incremental specification and verification of programs through the sound and principled application of static (at compile-time) and dynamic (at run-time) techniques—static techniques are applied where possible and dynamic ones where necessary. As a result, a gradual verifier allows developers to pick and choose which properties and components of their system to specify and receive immediate verification feedback without any unnecessary effort. Since the introduction of gradual verification [1], we have extended the approach to support more practical programs, such as those containing recursive heap data structures (trees, graphs, lists, etc.) [2, 3], and we have implemented a prototype [6, 7].

Student involvement
Because gradual verification is incremental, it requires a cost-benefit analysis to use effectively. When and where are the runtime penalties of gradual verification worth the cost of writing a full specification? How much easier is it, really, to write a gradual specification? Are gradual specifications just as useful for bug finding? Students who join us this summer will investigate these research questions using our existing prototype and will gain experience with programming language design, software development, theory, formal reasoning, logic, and empirical studies. Additionally, we intend for students who work with us this summer to present and/or publish their work at conferences (as a few students have done in the past: [2, 4, 5, 6, 7]).

[1] Johannes Bader, Jonathan Aldrich, and Éric Tanter. "Gradual Program Verification." In International Conference on Verification, Model Checking, and Abstract Interpretation, pp. 25-46. Springer, Cham, 2018.
[2] Jenna Wise, Johannes Bader, Cameron Wong, Jonathan Aldrich, Éric Tanter, and Joshua Sunshine. "Gradual verification of recursive heap data structures." Proceedings of the ACM on Programming Languages 4, no. OOPSLA (2020): 1-28.
[3] Jenna Wise, Johannes Bader, Jonathan Aldrich, Éric Tanter, and Joshua Sunshine. 2020. “Gradual Verification of Recursive Heap Data Structures.” First ACM SIGPLAN Workshop on Gradual Typing, January 25, 2020.
[4] Samuel Estep. "Gradual Program Analysis." Proceedings Companion of the 2019 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity. 2019.
[5] Samuel Estep, Jenna Wise, Jonathan Aldrich, Éric Tanter, Johannes Bader, and Joshua Sunshine. 2020. “Gradual Program Analysis.” First ACM SIGPLAN Workshop on Gradual Typing, January 25, 2020.
[6] Mona Zhang, and Jacob Gorenburg. "Design and implementation of a gradual verifier." In Companion Proceedings of the 2020 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity, pp. 31-33. 2020.
[7] Hemant Gouni and Conrad Zimmerman. 2021. “Implementation of an end-to-end gradual verification system.” In Companion Proceedings of the 2021 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH Companion 2021). Association for Computing Machinery, New York, NY, USA, 28–30.

Mentor: Christian Kästner

Description and significance
The advances in machine learning (ML) have stimulated widespread interest in integrating AI capabilities into various software products and services. Therefore today’s software development team often have both data scientists and software engineers, but they tend to have different roles. In an ML pipeline, there are in general two phases: an exploratory phase and a production phase. Data scientists commonly work in the exploratory phase to train an off-line ML model (often in computational notebooks) and then deliver it to software engineers who work in the production phase to integrate the model into the production codebase. However, data scientists tend to focus on improving ML algorithms to have better prediction results, often without thinking enough about the production environment; software engineers therefore sometimes need to redo some of the exploratory work in order to integrate it into production code successfully. In this project, we want to analyze collaboration between data scientists and software engineers, at technical and social levels, in open source and in industry.

Student involvement
We want to study how data scientists and software engineers collaborate. To this end, we will identify open source projects that use machine learning for production systems (e.g., Ubuntu's face recognition login) and study public artifacts or we will interview participants in production ML projects. This research involves interviews and analysis of software artifacts. We may also develop exploratory tools to define and document expectations and tests at the interface between different roles in a project. The project can be tailored to the students’ interests, but interests or a background in empirical methods would be useful. Familiarity with machine learning is a plus but not required. Note, this is not a data science/AI project, but a project on understanding *software engineering* practices relevant to data scientists.

MentorsVincent Hellendoorn and Rohan Padhye

Description and Significance
Writing tests is tedious, especially for large, complex systems. Tools that can generate them automatically are therefore a great asset, and these (esp. “fuzzers”) have indeed had great success at uncovering new bugs in critical systems. Unfortunately, the test cases they produce are often convoluted and inadequately human-readable. This makes debugging the discovered faults difficult, and ultimately the test tools themselves unusable. This lack of readability is mainly a lack of good metrics: fuzzers optimize for properties like coverage, compilability, and bug-detection potential. To deal with human-relevant properties, we need to turn to deep, machine learning. In this project, we will use language models on code [1] to automatically improve generated test cases’ readability, by using a model trained on the vast amount of test code available in the open source domain. This model can act as a guide to transform test cases, from renaming its variables or annotating tests with comments [4] to completely overhauling its structure, without losing their original quality.

Student Involvement
Students will first collect a large corpus of testable projects and their test suites from Github, for multiple languages such as Java, JavaScript, and C/C++. Students will then train large deep language models on their test code using state-of-the-art machine learning software frameworks and hardware. Students will augment one or more existing test generation tools, such as FuzzFactory [2], Evosuite [3], or C-Smith [5] incorporating predictability according to the learned models into their objective function, either instead of, or besides traditional metrics like code coverage. Students will also augment test-case minimization and debugging tools such as C-Reduce [6] to optimize for reducing cross-entropy along with input size. This will require developing new test case mutation operators such as variable renaming or nested-expression refactoring. Students will then perform an evaluation to measure the similarity between test cases auto-generated using these tools and human-authored tests. First, students will compare their predictability according to the language model using standard notions of cross-entropy. Then, students will conduct user studies with a small panel of human evaluators, who will be asked to rank manually authored and auto-generated test cases in terms of their readability. Finally, developers of open-source projects will be surveyed to evaluate whether auto-generated test cases using the techniques developed in this project are more preferable than those generated by traditional fuzzing techniques. A successful outcome of this project will be a set of tools that can produce automated tests which are indistinguishable from their human-authored counterparts, and that additionally aid developers in debugging.

[1] A. Hindle, E. T. Barr, Z. Su, M. Gabel, & P. Devanbu. On the naturalness of software. In Proceedings of the International Conference on Software Engineering (ICSE), 2012

[2] R. Padhye, C. Lemieux, K. Sen, L. Simon, and H. Vijayakumar. FuzzFactory: domain-specific fuzzing with waypoints. In Proceedings of the ACM on Programming Languages, (OOPSLA), 2019.
[3] Fraser, G. and Arcuri, A., Evosuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering (FSE), 2011.

[4] Roy, D., Zhang, Z., Ma, M., Arnaoudova, V., Panichella, A., Panichella, S., Gonzalez, D. & Mirakhorli, M. DeepTC-Enhancer: Improving the Readability of Automatically Generated Tests. In The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE) 2020.

[5] Yang, X., Chen, Y., Eide, E., & Regehr, J. (2011, June). Finding and understanding bugs in C compilers. In Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation (PLDI), 2011.

[6] Regehr, J., Chen, Y., Cuoq, P., Eide, E., Ellison, C., & Yang, X. Test-Case Reduction for C Compiler Bugs. In Proceedings of 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2012.

Mentors: Justine Sherry and James Hoe

Description and Significance
In this research we are building the next generation of operating systems optimized for high network throughput and low latency. Applications running on top of our new Linux-based operating system, Norman [1], can achieve network throughput as high as 100Gbps using a single CPU core.

There are two major aspects in the design of Norman that enable its great improvement in performance compared to the operating systems that we have today. First, Norman uses an FPGA-based network card to perform network operations traditionally implemented in the kernel. Second, Norman redesigns the interface between applications and the network card to reduce overhead on the software side, leaving more CPU cycles to applications
While Norman's use of an FPGA offers the opportunity to achieve high levels of parallelism when implementing network operations, it also requires major redesigns to data structures and algorithms in order to leverage this parallelism. For Norman to be adopted in the real world it needs to be able not only to achieve good performance but also to support existing Linux network tools such as tcpdump, iptables, and tc. As a result, these tools must also be redesigned to work on the FPGA. Moreover, Norman's use of a different interface between the network card and applications means that existing applications need to be adapted if they want to run as fast as possible under Norman.

Student Involvement
Students will get firsthand experience with a piece of Norman that they find most exciting. Students most interested in software will learn how to interact with network devices directly from user space and will work in adapting existing applications to work well with Norman. Students can explore different design tradeoffs in adapting these applications, such as throughput, latency and memory footprint. Students most interested in hardware will learn how to develop network functionality in hardware (in SystemVerilog). Students may choose to lead the development of a new feature that runs on the FPGA, such as a packet scheduler that works similarly to Linux traffic control or a flexible packet multiplexer that can be adapted to different applications.

[1] Hugo Sadok, Zhipeng Zhao, Valerie Choung, Nirav Atre, Daniel S. Berger, James C. Hoe, Aurojit Panda, and Justine Sherry. 2021. We need kernel interposition over the network dataplane. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS '21).

Mentor: Zhihao Jia

Description and significance
Recent years have witnessed the fast development of quantum computing, which hold significant advantages over classical algorithms on some algebraic problems, such as machine learning. In this project, we aim to develop an optimizing compiler for quantum applications. This project aims to tackle a fundamental challenge in building quantum compiler --- deploying quantum applications on modern quantum devices requires the compilers to consider a diversity of quantum gate sets and carry optimizations specific to different gate sets.

In this project, we will develop machine learning techniques to automatically discover potential quantum program optimizations for arbitrary quantum gate sets. The expected outcome of this project is an end-to-end ML-based quantum compiler that can potentially outperform existing heuristic compilers for quantum applications.

Student involvement
Students will get firsthand knowledge and experience on how machine learning techniques can be applied to optimize quantum programs. A REU student will be contributing to the development of an ML-based quantum program optimizer. The student will get the chance to explore and evaluate advanced ML techniques, such as reinforcement learning and graph neural network.

Mentors:  Joshua Sunshine, Keenan Crane, and Jonathan Aldrich

Description and significance
Every day, millions of professionals, teachers, and students in science, technology, engineering, and mathematics fields (STEM) work in the abstract world of mathematical structures, logical relationships, and computational processes. These abstractions are often presented in a textual and notation-heavy way, especially in formal literature, depriving readers of the powerful visual and spatial intuition that is fundamental to building deep understanding. People in STEM communicate informally by drawing diagrams, but they often omit these sketches in the literature because it takes a tremendous time of time and expertise to create mathematical diagrams [1]. Existing tools such as Mathematica, TikZ, and Adobe Illustrator lack the domain knowledge needed for creating mathematical diagrams, forcing the user to work at a low level.

Thus, our goal is to enable anyone to make beautiful diagrams by simply typing mathematical notation in plain text by building a platform called Penrose [2]. Penrose comprises three domain-specific languages, Domain, Substance, and Style, that are designed to enable users to write natural mathematical notation and separately specify the visual representation of notation in a domain, all in an extensible manner. Given a Domain, Substance, and Style program triple, the Penrose compiler transforms the problem of producing a diagram into a problem of numerical optimization, which is solved by an optimization runtime. The Penrose platform renders the solutions to the optimization problem as beautiful, automatic, custom mathematical diagrams.  

Student involvement
Students who work with us will join a highly interdisciplinary team spanning the fields of programming language design, software engineering, graphics, and visualization. Past students have had the opportunity to design and build fundamental parts of the Penrose platform, including the first Style compiler, a graphics API akin to “OpenGL for diagrams,” and an extensible syntactic sugar mechanism for mathematical notation. Future students will have many cross-cutting opportunities, including designing methods for automatic diagram optimization and layout, designing extensibility mechanisms for Substance and Style, working with mathematical domain experts to build Penrose's standard library, and conducting user studies on the usability of Domain, Substance, and Style. Students will gain experience in performing human-centered domain-specific language design, extending fundamental graphics methods such as optimization and sampling, and conducting interdisciplinary research.

[1] Ma'ayan, D., Ni, W., Ye, K., Kulkarni, C., & Sunshine, J. (2020, April). How Domain Experts Create Conceptual Diagrams and Implications for Tool Design. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-14).

[2] Ye, K., Ni, W., Krieger, M., Ma'ayan, D., Wise, J., Aldrich, J., ... & Crane, K. (2020). Penrose: from mathematical notation to beautiful diagrams. ACM Transactions on Graphics (TOG), 39(4), 144-1.

Mentors: Hanan Hibshi and Maverick Woo

Description and Significance
picoCTF is a free online Capture The Flag-style game implemented by and maintained at Carnegie Mellon University to promote and facilitate cybersecurity education. The target audience of our service has been middle and high school students since 2013, but it has also been increasingly popular and attracted many other age groups, including college students and adult learners. Each picoCTF release features ~100 cybersecurity challenge problems of increasing difficulty, which are revealed over a storyline to guide players to hone their skills one step at a time. These security challenges cover concepts such as cryptography, binary exploitation, web exploitation, reverse engineering, forensics, and other topics related to security and privacy. As a year-round education platform, the picoCTF team conducts constant research in areas including cybersecurity education methodologies, CTF problem development, and education platform development. This summer, we have multiple research projects for motivated students to get involved in various missions in picoCTF:

(1) Collect empirical data through surveys, user studies, and classroom observations To improve the design of the picoCTF platform to reach a larger number of students, especially in under-resourced communities, we need to collect and analyze empirical data to inform our design enhancement and platform scalability process. This project includes a research plan to obtain the needed data through conducting user studies, focus groups, and usability and scalability tests that examine picoCTF in a classroom setting. We are interested in understanding how we can enhance our platform to better serve K-12 teachers and students, college students, and independent adult learners in under-resourced communities. The empirical data collected by closely observing participants in focus groups and from surveys and questionnaires will provide informed feedback to the picoCTF research team about possible technical challenges and key design improvements that would enhance the students experience.

(2) Analyze player data from previous years and develop visualization tools In addition to the surveys and questionnaires, the current picoCTF platform is rich with player data from previous years that could be a valuable resource to conduct in-depth analysis that would help understand how to improve the game to reach a wider audience, and what/where to include new challenges. The analysis would help reveal patterns that could be mapped to educational goals and help investigate where players fail to solve challenges or where the game becomes less interesting. These findings could ultimately help improve the user experience and retainment. Other areas of analysis include performance by gender, team diversity, age, educational background, etc. We envision students will learn to use modern data analysis toolkits to analyze our data and build an interactive web-based exploration tool for presenting the findings from these analyses.

(3) Write new CTF challenges for the game and test current CTF challenges Writing and testing CTF challenges are ongoing tasks in picoCTF. Testing current challenges help identify errors and bugs before a future competition goes live, and writing new challenges help increase our challenges pool. For the latter, we are especially interested in new challenges in areas that we have minimal or no existing coverage on our platform. These include privacy, mobile security (Android / iOS), IoT security (e.g., embedded Linux-based device), and ICS security (e.g., RTOS, ROS). Students will engage in learning security and privacy problems that arise in these areas and develop new CTF challenges of gradually increasing complexity to cater to players with different stages of technical capability.

Student involvement
We are looking for multiple students who are interested in cybersecurity and who enjoy working on projects that have a global impact on youth and workforce development. Dr. Hanan Hibshi and Dr. Maverick Woo will be the faculty mentors overseeing students research activities, and the picoCTF team consists of software engineers and graduate students who work together with student research assistants. The picoCTF project is interdisciplinary in nature and can be attractive to students with different backgrounds. Students with Human Computer Interaction background can enjoy conducting user studies, collecting the empirical data, or examining the picoCTF interface and propose design changes that can improve user experience. Students with interest in CS education and/or cognitive psychology could help analyze the data from existing players to investigate ways that can improve learning outcomes. Students who enjoy software development can help the technical team improve the current gaming platform and migrate into the new 2021 picoCTF that has advanced features. Finally, students with cybersecurity background can join our team and enjoy testing the current challenges (by playing the game!) and help create new challenges or add new category of challenges.

Mentors: Nathan Beckmann

Description and Significance
This project is about how to improve the hardware-software interface to give software better visibility and control over data movement. Moving data is much more expensive than computation, but this reality is not reflected in the current hardware-software interface. In current systems, software simply reads and writes data, and hardware (e.g., caches) move data around to perform the requested reads and writes. Software can't tell where data is or when it moves. As a result, programs run unnecessarily inefficiently, and there is no good way to fix the problem in software today. We are exploring ways to expand the hardware-software interface that let software see when data moves and trigger computation in response. This new interface opens the door to many features and optimizations that eliminate data movement, reduce work, improve security, etc.

Student involvement
This project is looking for students with an interest in VLSI, computer architecture, and/or compilers.

Mentor: Steven Wu

Many modern applications of machine learning (ML) rely on datasets that may contain sensitive personal information, including medical records, browsing history, and geographic locations. To protect the private information of individual citizens, many ML systems now train their models subject to the constraint of differential privacy (DP), which informally requires that no individual training example has a significant influence on the trained model. After well over a decade of intense theoretical study, DP has recently been deployed by many organizations, including Microsoft, Google, Apple, LinkedIn, and more recently the 2020 US Census. However, the majority of the existing practical deployments still focus on rather simple data analysis tasks (e.g., releasing simple counts and histogram statistics). To put DP to practice for more complex machine learning tasks, this project will study new differentially private training methods for deep learning that improve on existing state-of-the-art methods. We will also study how to use DP deep learning techniques to train deep generative models, which can generate privacy-preserving synthetic data—a collection of “fake” data that preserve important statistical properties of the original private data set. This, in turn, will enable privacy-preserving data sharing.

Mentor: Eunsuk Kang

Description and significance
Unintuitive, badly designed human-machine interfaces (HMI) are not merely an annoyance, but can pose significant risks to users in systems where safety is a key concern. For example, poor HMI design has been attributed as a major factor in numerous fatal accidents involving medical devices, automobiles, and aviation systems. The challenge is that humans are far from perfect and inadvertently make mistakes from time to time, but many interfaces are not designed to deal with such human errors; as a result, when a safety failure occurs, it is often the users who get blamed, even when a better designed interface could have prevented such a failure. To tackle this challenge, Dr. Kang is collaborating with researchers in human factors and cognitive science to systematically design HMIs that are robust against human errors [1]; i.e., an interface that is explicitly designed to recognize and handle potential human errors and prevent them from causing safety failures.

Student involvement
In this project, students will contribute to the development of a new methodology for designing safe and robust HMIs. Potential research tasks include: (1) developing a theoretical foundation for robust HMIs, (2) a mathematical approach for modeling human behavior and errors, (3) an automated technique for analyzing an HMI for its robustness against human errors, (4) an automated technique for repairing an HMI to improve its robustness, and (5) designing and running a user study to evaluate the HMIs. Students will also get a chance to apply these techniques to realistic case studies, including interfaces for medical devices and intelligent vehicles. Students are expected to have a basic background in logic and discrete math. Knowledge in formal methods, programming languages and/or HCI is a plus.

[1] NSF project description: Preventing Human Errors in Cyber-human Systems with Formal Approaches to Human Reliability Rating and Model Repair 

Mentor: Eunsuk Kang

Description and significance
Software is increasingly being used to control systems that closely interact with physical entities in the world, and whose failures may result in irreversible damage to users and our environment. These so called cyber-physical systems (CPSs), ranging from autonomous vehicles and drones to medical devices, pose new challenges in software engineering. Unlike traditional software applications, these systems are deployed in a highly dynamic, uncertain environment, and often rely on fragile assumptions about the behavior of users and other environmental agents (some of which may be malicious). Due to this dynamic, adversarial nature, software in CPSs must be designed to handle unexpected interactions with the environment, and guarantee an acceptable level of safety and security even in presence of component failures. Dr. Kang and his collaborators are working on (1) new principles and techniques for designing robust software [1,2], (2) algorithms and tools for reasoning about critical properties of CPSs [3,4].

Student involvement
In this project, students will develop new techniques for designing safe and secure CPSs. Potential research tasks include developing (1) a language for specifying the behavior of a complex CPS at a high-level of abstraction, (2) a technique for automatically analyzing a system for potential vulnerabilities, and (3) an architectural mechanism for guaranteeing safety and security even in presence of unexpected inputs. Students will also get an hands-on experience applying these techniques to real-world case studies, such as autonomous drones, intelligent vehicles, and medical devices. Students are expected to have a basic background in logic and discrete mathematics, and will learn about the-state-of-the-art techniques in formal methods, CPS design, software analysis, and programming languages.

[1] Property-Driven Runtime Resolution of Feature Interactions. Santhana Gopalan Raghavan, Kosuke Watanabe, Eunsuk Kang, Chung-Wei Lin, Zhihao Jiang, and Shinichi Shiraishi. International Conference on Runtime Verification (RV), 2018.
[2] An Architectural Mechanism for Resilient IoT Services. Hokeun Kim, Eunsuk Kang, Edward A. Lee, and David Broman. ACM Workshop on the Internet of Safe Things (SafeThings), 2017.
[3] Model-Based Security Analysis of a Water Treatment System. Eunsuk Kang, Sridhar Adepu, Daniel Jackson, and Aditya P. Mathur. ICSE Workshop on Smart Cyber-Physical Systems, 2016.
[4] Multi-Representational Security Analysis. Eunsuk Kang, Aleksandar Milicevic, and Daniel Jackson. Symposium on the Foundations of Software Engineering (FSE), 2016.

Mentors: David Garlan and Bradley Schmerl

Description and significance
Self-adaptive systems (e.g., automotive or robotics systems) are often intended to be used in a variety of contexts that are characterized by different weather conditions, the state of the system, user behavior, or other environmental factors [2]. While these systems are commonly designed to operate autonomously, it is important to elicit human stakeholders' preferences and constraints (e.g., to comply with regulations and meet stakeholders’ needs). In previous research, a negotiation support system has been developed [1] that allows humans to express and negotiate their preferences and constraints and uses their input to define a “utility function” that can be used by a self-adaptive system. So far, it is not possible to specify the contexts/situations that these preferences and constraints should hold in. The goal of this project is to address this gap and support the elicitation of contextual requirements for self-adaptive systems.

Student involvement
Students will be involved in designing solutions to address the described research gap, for example, in defining what contextual aspects should be considered when describing requirements for self-adaptive systems, developing a conceptual model for systems' contexts, and extending the negotiation support system’s user interface. The ideal student has an interest in self-adaptive systems, modeling, and user interface design.

[1] Wohlrab, R., & Garlan, D. A Negotiation Support System for Defining Utility Functions for Multi-Stakeholder Self-Adaptive Systems. Accepted to Requirements Engineering.

[2] Alegre, U., Augusto, J. C., & Clark, T. (2016). Engineering context-aware systems and applications: A survey. Journal of Systems and Software, 117, 55-83.


Mentors: Christian Kästner and Bogdan Vasilescu

Description and significance
Reuse of open source artifacts in software ecosystems has enabled significant advances in development efficiencies as developers can now build on significant infrastructure and develop apps or server applications in days rather than months or years. However, despite its importance, maintenance of this open source infrastructure is often left to few volunteers with little funding or recognition, threatening the sustainability of individual artifacts, such as OpenSSL, or entire software ecosystems. Reports of stress and burnout among open source developers are increasing. The teams of Dr. Kaestner and Dr. Vasilecu have explored dynamics in software ecosystems to expose differences, understand practices, and plan interventions [1,2,3,4]. Results indicate that different ecosystems have very different practices and interventions should be planned accordingly [1], but also that signaling based on underlying analyses can be a strong means to guide developer attention and affect change [2]. This research will further explore sustainability challenges in open source with particular attention to the interaction between paid and volunteer contributors and stress and resulting turnover.

Student involvement
Students will empirical study sustainability problems and interventions, using interviews, surveys, and statistical analysis of archival data (e.g., regression modeling, time series analysis for causal inference). What are the main reasons for volunteer contributors to drop out of open source projects? In what situations do volunteer contributors experience stress? In which projects will other contributors step up and continue maintenance when the main contributors leave? Which past interventions, such as contribution guidelines and code of conducts, have been successful in retaining contributors and easing transitions? How to identify subcommunities within software ecosystems that share common practices and how do communities and subcommunities learn from each other? Students will investigate these questions by exploring archival data of open source development traces (, will design interviews or surveys, will apply statistical modeling techniques, will build and test theories, and conduct literature surveys. Students will learn state of the art research methods in empirical software engineering and apply them to specific sustainability challenges of great importance. Students will actively engage with the open source communities and will learn to communicate their results to both academic and nonacademic audiences.

[1] Christopher Bogart and Christian Kästner and James Herbsleb and Ferdian Thung. How to Break an API: Cost Negotiation and Community Values in Three Software Ecosystems. In Proc. Symposium on the Foundations of Software Engineering (FSE), 2016.

[2] Asher Trockman, Shurui Zhou, Christian Kästner, and Bogdan Vasilescu. Adding sparkle to social coding: an empirical study of repository badges in the npm ecosystem. In Proc. International Conference on Software Engineering (ICSE), 2018.

[3] Bogdan Vasilescu, Kelly Blincoe, Qi Xuan, Casey Casalnuovo, Daniela Damian, Premkumar Devanbu, and Vladimir Filkov. The sky is not the limit: multitasking across github projects. In Proc. International Conference on Software Engineering (ICSE), 2016.

[4] Bogdan Vasilescu, Daryl Posnett, Baishakhi Ray, Mark GJ van den Brand, Alexander Serebrenik, Premkumar Devanbu, and Vladimir Filkov. Gender and tenure diversity in GitHub teams. In Proc. ACM Conference on Human Factors in Computing Systems (CHI), 2015.

Mentor: Lorrie Cranor

Description and Significance
Many websites offer visitors privacy controls and opt-out choices, either to comply with legal requirements or to address consumer privacy concerns. The way these control mechanisms are implemented can significantly affect individuals’ choices and their privacy outcomes. We have conducted research to survey and evaluate existing controls on websites (e.g. cookie consent banners) and apps and are designing prototype mechanisms to improve experiences related to consent and privacy. We are also taking this beyond the web to examine consent experiences for AR/VR, IoT, and audio devices. There is interest both from regulators and companies in developing guidance for usable consent, opt-out, and privacy choices interfaces.

Student Involvement
The student will work with the research team to survey existing consent experiences in a particular domain or service (e.g., Facebook, health apps, AR glasses), prototype changes to privacy and consent related controls, and/or conduct online user studies to test their effectiveness. Specific domains and focus of study will be determined at the start of the summer. Students should have an interest in privacy and user study design. Prior exposure to HCI, such as an undergraduate HCI or interaction design course is helpful but not required.

Hana Habib, Yixin Zou, Yaxing Yao, Alessandro Acquisti, Lorrie Cranor, Joel Reidenberg, Norman Sadeh, and Florian Schaub. 2021. Toggles, Dollar Signs, and Triangles: How to (In)Effectively Convey Privacy Choices with Icons and Link Texts. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 63, 1–25.

Hana Habib, Sarah Pearman, Jiamin Wang, Yixin Zou, Alessandro Acquisti, Lorrie Faith Cranor, Norman Sadeh, and Florian Schaub. “It’s a scavenger hunt”: Usability of Websites’ Opt-Out and Data Deletion Choices. CHI 2020.

Hana Habib, Yixin Zou, Aditi Jannu, Neha Sridhar, Chelse Swoopes, Alessandro Acquisti, Lorrie Faith Cranor, Norman Sadeh, and Florian Schaub. An Empirical Analysis of Data Deletion and Opt-Out Choices on 150 Websites. SOUPS 2019.


Mentor: Daniel Klug

Description and significance
Short-form video apps, foremost TikTok, are currently the most popular social media platforms among younger people. Their success is largely based on their high accessibility and ubiquitousness in regards to online social interaction and participation. But a key element of TikTok is the app’s specific, yet, mysterious algorithm that caters individual video feeds for users based on their content consumption and browsing behavior. While first studies are looking to analyze the TikTok algorithm and some basic knowledge exists about it, we have only little understanding about what social media users know about socio-technical aspects of short-video apps when they create, share, and redistrubute video content. In the MINT Lab (, we are using qualitative approaches, such as interviews and content analysis, to research users’ opinions, knowledge, and awareness in using highly popular short-video apps for communication, socialization, and entertainment. Possible research questions are: What are common user understandings of the TikTok algorithm? What are users’ ways of observing how the algorithm might work? How does algorithmically generated content influence users' perception of online and offline reality? What is the extent of users' algorithmic literacy and how does it influence their online behavior and interaction? The goal is to study and understand how humans as users interact with social technology and how the use of social media apps is connected to and integrated into our everyday life.

Student involvement
Students will learn how to design qualitative research projects and how to apply qualitative methods, such as interviews, user studies, or content analysis, to research socio-technological aspects of social media use and engagement. For example, designing interview questions, finding and contacting potential interviewees, best practices for conducting interviews with users, and how to transcribe, code, analyze, and interpret interviews. Based on quality criteria for qualitative research, students will learn how to develop and validate hypotheses from qualitative data and how to apply qualitative results to mixed-method designs. The ideal student is familiar with social media platforms, has interest in qualitative research, and is open to conduct interviews.

Klug, D., Qin, Y., Evans, M., & Kaufman, G. (2021, June). Trick and Please. A Mixed-Method Study On User Assumptions About the TikTok Algorithm. In 13th ACM Web Science Conference 2021 (pp. 84-92).

De Los Santos, M. & Klug, D. (2021, December). The TikTok Tradeoff: Compelling Algorithmic Content at the Expense of Personal Privacy. In 20th International Conference on Mobile and Ubiquitous Multimedia (MUM 2021).