Interest Form for Participating in the AI Researcher Human Study (Execution Stage)

JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

Hi! We are a team of researchers from the Stanford NLP group: Chenglei Si, Tatsunori Hashimoto, and Diyi Yang.

About the Study:

We are conducting a human study to compare AI with human experts on generating research ideas on NLP research.

Qualifications:

We are looking for researchers with past NLP research experiences, for example, as demonstrated by at least one co-authored publication at a top-tier NLP/AI conference or journal. We will perform some basic screening based on your responses and notify you if we decide to invite you to our study. Note that we will be recruiting in batches so we might send out invitation emails on a rolling basis over the next two months.

Task and Compensation:

Our task involves executing a research idea into a full project (including writing the code implementation and conducting the specified experiments). Thus, we expect a workload of around 40+ hours spanning 1-2 months in total. We will compensate your time by paying an hourly rate of at least $20, plus a heavy completion bonus and quality bonus. In total, you will be compensated $3000 - $5000 if you finish the whole execution task. We will do some periodic check-ins with you throughout your project execution process. Note that we will provide all necessary computation resources needed in executing your assigned idea (e.g., OpenAI, Anthropic, Together credits), as well as human annotation budgets if needed (e.g., AMT, Prolific, Upwork).

Intellectual Property:

We will collect your implemented codebase and experiment results as part of our final dataset. However, you are allowed to submit the resultant paper from your executed project to any conference or journal if you wish, and you will be the lead author on your executed paper (you don't need to include us as co-authors).

Next Steps:

You should fill in the form if you are interested in the study and are willing to commit. We will assign tasks to you based on your indicated qualification and available time. We will contact you by email if we decide to recruit you as a participant. Once you sign the consent form, we will send over the task instructions for the corresponding task. Note that submitting this form will put you in our pool of interested participants, and we will recruit and contact participants on a rolling basis over the next two months.

This project has been approved by Stanford Institutional Review Board (IRB# 74246). You can contact Chenglei Si (clsi@stanford.edu) for any questions or concerns.

Your Email *

Your Name *

Your Academic Affiliation *

Which country are you located in? *

Your Job (If student, please indicate which year). *

How many hours can you spend on the study over the next six months? *

Please select your earliest start date. *

YYYY

Among the following research topics, please select all that you are interested in and feel qualified to work on. *

Coding: novel prompting methods for large language models to improve code generation

Math: novel prompting methods for large language models to improve mathematical problem solving

Factuality: novel prompting methods that can improve factuality and reduce hallucination of large language models

Bias: novel prompting methods to reduce social biases and stereotypes of large language models

Safety: novel prompting methods to improve large language models' robustness against adversarial attacks or improve their security or privacy

Multilingual: novel prompting methods to improve large language models’ performance on multilingual tasks or low-resource languages and vernacular languages

Uncertainty: novel prompting methods that can better quantify uncertainty or calibrate the confidence of large language models

Required

Can you provide your academic profile? (preferably Google Scholar or Semantic Scholar; if not you can write a paragraph to describe your research experiences) *

If you have an academic homepage, you can share it here for us to better understand your research background.

Submit

Clear form

Never submit passwords through Google Forms.

This content is neither created nor endorsed by Google. - Terms of Service - Privacy Policy

Does this form look suspicious? Report

Forms