Study on LLMs and Developers

We invite you to participate in a research study being conducted at the Colorado School of Mines. We are studying how people with different levels of programming experience complete software-development tasks and use online resources, including AI coding tools. What you will do You will complete three short programming tasks involving C code: Read and comment on existing code Fix code so that it is functionally correct and secure Write a small function You will complete the tasks in a secure browser-based study environment. After the programming portion, you will complete a follow-up survey about your experience. The programming portion is limited to 60 minutes. The follow-up survey is expected to take approximately 15–30 minutes. You do not need to complete every task correctly to receive the base compensation. You may move forward or skip a task when necessary. Study conditions Participants may be assigned to a condition that either permits or prohibits the use of AI coding assistants. Your assigned instructions will be clearly displayed before the study begins. You must follow the rules for your assigned condition and use only the tools available through the study environment. Compensation Compensation is structured as follows: $10 for a qualifying good-faith attempt An additional $10 for fully completing the study A potential $5 correctness bonus for each programming task, up to three tasks The maximum possible compensation is: $20 for full study participation Up to $15 in correctness bonuses Maximum total: $35 A qualifying good-faith attempt means that you enter the study environment, follow the study instructions, and make a genuine effort on at least one programming task. Merely opening the study link and immediately leaving does not qualify. Full completion means that you: Proceed through all three programming tasks, completing or skipping each as needed Reach and complete the final survey Send us the completion code shown at the end of the survey Correctness bonuses will be determined after the submitted task solutions are evaluated against predetermined correctness criteria. Base compensation does not depend on producing correct solutions. Participation requirements You should: Be at least 18 years old Be able to complete the study on a desktop or laptop computer Have a stable internet connection Be able to complete the programming portion in one sitting Follow the assigned rules concerning AI-tool use Complete the study independently Not participate more than once Programming experience is helpful, but applicants with different experience levels may be eligible. Please do not begin the study until you have received and accepted an Upwork offer. Eligible applicants will receive a private consent and study link through Upwork Messages. Not every applicant will necessarily be selected. Thank you for helping with our research.