Initial Commit

Delete Redundant Files
Scripts for Download and Process Submission
Improve Data Loading

Add initial app

.gitignore

@@ -0,0 +1,185 @@
+
+
+key.py
+
+*.pdf
+*.json
+*.png
+*.jpg
+*.jpeg
+*.gif
+data/
+unused_data/
+demo/
+Summary/
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+outputs
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+.idea/
+
+.DS_Store
+hf-spaces/
+etc/
+.conda
+*.xlsx
+*.csv
+*.zip

LICENSE

@@ -0,0 +1,203 @@
+Copyright 2023 ChatArena. All rights reserved.
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -1,14 +1,91 @@
+# AgentReview
+
+Official implementation for the 🔗[EMNLP 2024](https://2024.emnlp.org/) (main) paper: [AgentReview: Exploring Peer Review Dynamics with LLM Agents](https://arxiv.org/abs/2406.12708)
+
+* 🌐 Website: [https://agentreview.github.io/](https://agentreview.github.io/)
+* 📄 Paper: [https://arxiv.org/abs/2406.12708](https://arxiv.org/abs/2406.12708)
+* **🚀 Note: This repository is under construction development. Please stay tuned!!**
+
+
+
+```bibtex
+@inproceedings{jin2024agentreview,
+  title={AgentReview: Exploring Peer Review Dynamics with LLM Agents},
+  author={Jin, Yiqiao and Zhao, Qinlin and Wang, Yiyang and Chen, Hao and Zhu, Kaijie and Xiao, Yijia and Wang, Jindong},
+  booktitle={EMNLP},
+  year={2024}
+}
+```
+
+<img src="static/img/Overview.png">
+
 ---
-title: AgentReview
-emoji: 👁
-colorFrom: indigo
-colorTo: pink
-sdk: gradio
-sdk_version: 5.4.0
-app_file: app.py
-pinned: false
-license: apache-2.0
-short_description: EMNLP 2024
----
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+### Introduction
+
+Agent4Review is a simulation framework for systematic analysis of the peer review process based on large language models (LLMs). This framework aims to understand decision-making patterns, reviewer behavior, and the dynamics of paper acceptance and rejection.
+
+### Academic Abstract
+
+Peer review is fundamental to the integrity and advancement of scientific publication. Traditional methods of peer review analyses often rely on exploration and statistics of existing peer review data, which do not adequately address the multivariate nature of the process, account for the latent variables, and are further constrained by privacy concerns due to the sensitive nature of the data. We introduce AgentReview, the first large language model (LLM) based peer review simulation framework, which effectively disentangles the impacts of multiple latent factors and addresses the privacy issue. Our study reveals significant insights, including a notable 37.1% variation in paper decisions due to reviewers' biases, supported by sociological theories such as the social influence theory, altruism fatigue, and authority bias. We believe that this study could offer valuable insights to improve the design of peer review mechanisms
+
+
+![Review Stage Design](static/img/ReviewPipeline.png)
+
+
+### Installation
+
+1. Download and unzip the [data](https://www.dropbox.com/scl/fi/mydblhx8yxk8kbz8b7zmr/AgentReview_data.zip?rlkey=se16p9gonclw5t8t3vn9p0o6n&st=6988u8lx&dl=0) under `data/`, which contains the PDF versions of the paper as well as the forum discussions for ICLR 2020 - 2023
+2. **Install Required Packages**:
+   ```
+   cd AgentReview
+   pip install -r requirements.txt
+   ```
+   
+3. **Run the Project**:
+
+   **Note: all project files should be run from the `AgentReview` directory.**
+    
+
+## Data
+
+#### Project Structure
+- `app.py`: The main application file for running the framework.
+- `analysis/`: Contains Python scripts for various statistical analyses of review data.
+- `chatarena/`: Core module for simulating different review environments and integrating LLM backends.
+- `dataset/`: Scripts for handling dataset operations, such as downloading and processing submissions.
+- `demo/`: Demonstrative scripts showcasing the functionality of different components.
+- `docs/`: Documentation files and markdown guides for using and extending the framework.
+- `examples/`: Configuration files and examples to demonstrate the capabilities and setup of simulations.
+- `experiments/`: Experimental scripts to test new ideas or improvements on the framework.
+- `visual/`: Visualization scripts for generating insightful plots and charts from the simulation data.
+
+#### Usage
+
+**[UNDER CONSTRUCTION]**
+
+
+### Stage Design
+
+Our simulation adopts a structured, 5-phase pipeline
+
+* **Phase I. Reviewer Assessment.** Each manuscript is evaluated by three reviewers independently.
+* **Phase II. Author-Reviewer Discussion.** Authors submit rebuttals to address reviewers' concerns;
+* **Phase III. Reviewer-AC Discussion.** The AC facilitates discussions among reviewers, prompting updates to their initial assessments.
+* **Phase IV. Meta-Review Compilation.** The AC synthesizes the discussions into a meta-review.
+* **Phase V. Paper Decision.** The AC makes the final decision on whether to accept or reject the paper, based on all gathered inputs.
+
+## Note
+
+- We use a fixed acceptance rate of 32%, corresponding to the actual acceptance rate of ICLR 2020 -- 2023. See [Conference Acceptance Rates](https://github.com/lixin4ever/Conference-Acceptance-Rate) for more information.
+- Sometimes the API can apply strict filtering to the request. You may need to adjust the content filtering to get the desired results.  
+
+
+
+## License
+
+This project is licensed under the Apache-2.0 License.
+
+## Acknowledgements
+
+The implementation is partially based on the [chatarena](https://github.com/Farama-Foundation/chatarena) framework.

agentreview/__init__.py

@@ -0,0 +1,8 @@
+import os
+
+ROOT_DIR = (
+    os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir)) + os.path.sep
+)
+EXAMPLES_DIR = os.path.join(ROOT_DIR, "examples")
+
+__version__ = "0.1.16"

agentreview/agent.py

@@ -0,0 +1,253 @@
+import logging
+import re
+import uuid
+from abc import abstractmethod
+from argparse import Namespace
+from typing import List, Union
+
+from tenacity import RetryError
+
+from .backends import IntelligenceBackend, load_backend
+from .config import AgentConfig, BackendConfig, Configurable
+from .message import SYSTEM_NAME, Message
+
+# A special signal sent by the player to indicate that it is not possible to continue the conversation, and it requests to end the conversation.
+# It contains a random UUID string to avoid being exploited by any of the players.
+SIGNAL_END_OF_CONVERSATION = f"<<<<<<END_OF_CONVERSATION>>>>>>{uuid.uuid4()}"
+
+
+class Agent(Configurable):
+    """An abstract base class for all the agents in the chatArena environment."""
+
+    @abstractmethod
+    def __init__(
+        self, name: str, role_desc: str, global_prompt: str = None, *args, **kwargs
+    ):
+        """
+        Initialize the agent.
+
+        Parameters:
+            name (str): The name of the agent.
+            role_desc (str): Description of the agent's role.
+            global_prompt (str): A universal prompt that applies to all agents. Defaults to None.
+        """
+        super().__init__(
+            name=name, role_desc=role_desc, global_prompt=global_prompt, **kwargs
+        )
+        self.name = name
+        self.role_desc = role_desc
+        self.global_prompt = global_prompt
+
+
+class Player(Agent):
+    """
+    The Player class represents a player in the chatArena environment.
+
+    A player can observe the environment
+    and perform an action (generate a response) based on the observation.
+    """
+
+    def __init__(
+        self,
+        name: str,
+        role_desc: str,
+        backend: Union[BackendConfig, IntelligenceBackend],
+        global_prompt: str = None,
+        args: Namespace = None,
+        **kwargs,
+    ):
+        """
+        Initialize the player with a name, role description, backend, and a global prompt.
+
+        Parameters:
+            name (str): The name of the player.
+            role_desc (str): Description of the player's role.
+            backend (Union[BackendConfig, IntelligenceBackend]): The backend that will be used for decision making. It can be either a LLM backend or a Human backend.
+            global_prompt (str): A universal prompt that applies to all players. Defaults to None.
+        """
+
+        self.data_dir = kwargs.pop("data_dir", None)
+        self.args = args
+
+        if isinstance(backend, BackendConfig):
+            backend_config = backend
+            backend = load_backend(backend_config)
+        elif isinstance(backend, IntelligenceBackend):
+            backend_config = backend.to_config()
+        else:
+            raise ValueError(
+                f"backend must be a BackendConfig or an IntelligenceBackend, but got {type(backend)}"
+            )
+
+        assert (
+            name != SYSTEM_NAME
+        ), f"Player name cannot be {SYSTEM_NAME}, which is reserved for the system."
+
+        # Register the fields in the _config
+        super().__init__(
+            name=name,
+            role_desc=role_desc,
+            backend=backend_config,
+            global_prompt=global_prompt,
+            **kwargs,
+        )
+
+        self.backend = backend
+
+    def to_config(self) -> AgentConfig:
+        return AgentConfig(
+            name=self.name,
+            role_desc=self.role_desc,
+            backend=self.backend.to_config(),
+            global_prompt=self.global_prompt,
+        )
+
+    def act(self, observation: List[Message]) -> str:
+        """
+        Take an action based on the observation (Generate a response), which can later be parsed to actual actions that affect the game dynamics.
+
+        Parameters:
+            observation (List[Message]): The messages that the player has observed from the environment.
+
+        Returns:
+            str: The action (response) of the player.
+        """
+        try:
+            response = self.backend.query(
+                agent_name=self.name,
+                role_desc=self.role_desc,
+                history_messages=observation,
+                global_prompt=self.global_prompt,
+                request_msg=None,
+            )
+        except RetryError as e:
+            err_msg = f"Agent {self.name} failed to generate a response. Error: {e.last_attempt.exception()}. Sending signal to end the conversation."
+            logging.warning(err_msg)
+            response = SIGNAL_END_OF_CONVERSATION + err_msg
+
+        return response
+
+    def __call__(self, observation: List[Message]) -> str:
+        return self.act(observation)
+
+    async def async_act(self, observation: List[Message]) -> str:
+        """
+        Async version of act().
+
+        This is used when you want to generate a response asynchronously.
+
+        Parameters:
+            observation (List[Message]): The messages that the player has observed from the environment.
+
+        Returns:
+            str: The action (response) of the player.
+        """
+        try:
+            response = self.backend.async_query(
+                agent_name=self.name,
+                role_desc=self.role_desc,
+                history_messages=observation,
+                global_prompt=self.global_prompt,
+                request_msg=None,
+            )
+        except RetryError as e:
+            err_msg = f"Agent {self.name} failed to generate a response. Error: {e.last_attempt.exception()}. Sending signal to end the conversation."
+            logging.warning(err_msg)
+            response = SIGNAL_END_OF_CONVERSATION + err_msg
+
+        return response
+
+    def reset(self):
+        """
+        Reset the player's backend in case they are not stateless.
+
+        This is usually called at the end of each episode.
+        """
+        self.backend.reset()
+
+
+class Moderator(Player):
+    """
+    The Moderator class represents a special type of player that moderates the conversation.
+
+    It is usually used as a component of the environment when the transition dynamics is conditioned on natural language that are not easy to parse programmatically.
+    """
+
+    def __init__(
+        self,
+        role_desc: str,
+        backend: Union[BackendConfig, IntelligenceBackend],
+        terminal_condition: str,
+        global_prompt: str = None,
+        **kwargs,
+    ):
+        """
+        Initialize the moderator with a role description, backend, terminal condition, and a global prompt.
+
+        Parameters:
+            role_desc (str): Description of the moderator's role.
+            backend (Union[BackendConfig, IntelligenceBackend]): The backend that will be used for decision making.
+            terminal_condition (str): The condition that signifies the end of the conversation.
+            global_prompt (str): A universal prompt that applies to the moderator. Defaults to None.
+        """
+        name = "Moderator"
+        super().__init__(
+            name=name,
+            role_desc=role_desc,
+            backend=backend,
+            global_prompt=global_prompt,
+            **kwargs,
+        )
+
+        self.terminal_condition = terminal_condition
+
+    def to_config(self) -> AgentConfig:
+        return AgentConfig(
+            name=self.name,
+            role_desc=self.role_desc,
+            backend=self.backend.to_config(),
+            terminal_condition=self.terminal_condition,
+            global_prompt=self.global_prompt,
+        )
+
+    def is_terminal(self, history: List[Message], *args, **kwargs) -> bool:
+        """
+        Check whether an episode is terminated based on the terminal condition.
+
+        Parameters:
+            history (List[Message]): The conversation history.
+
+        Returns:
+            bool: True if the conversation is over, otherwise False.
+        """
+        # If the last message is the signal, then the conversation is over
+        if history[-1].content == SIGNAL_END_OF_CONVERSATION:
+            return True
+
+        try:
+            request_msg = Message(
+                agent_name=self.name, content=self.terminal_condition, turn=-1
+            )
+            response = self.backend.query(
+                agent_name=self.name,
+                role_desc=self.role_desc,
+                history_messages=history,
+                global_prompt=self.global_prompt,
+                request_msg=request_msg,
+                *args,
+                **kwargs,
+            )
+        except RetryError as e:
+            logging.warning(
+                f"Agent {self.name} failed to generate a response. "
+                f"Error: {e.last_attempt.exception()}."
+            )
+            return True
+
+        if re.match(
+            r"yes|y|yea|yeah|yep|yup|sure|ok|okay|alright", response, re.IGNORECASE
+        ):
+            # print(f"Decision: {response}. Conversation is ended by moderator.")
+            return True
+        else:
+            return False

agentreview/arena.py

@@ -0,0 +1,201 @@
+import csv
+import json
+import logging
+import uuid
+from typing import Dict, List, Union
+
+from .agent import Player
+from .backends import Human
+from .config import ArenaConfig
+from .environments import Environment, TimeStep, load_environment
+
+
+class TooManyInvalidActions(Exception):
+    pass
+
+
+class Arena:
+    """Utility class that manages the game environment and players."""
+
+    def __init__(
+        self, players: List[Player], environment: Environment, args,  global_prompt: str = None
+    ):
+        # Create a container for the players and environment and reset the game
+        self.players = players
+        self.environment = environment
+        self.global_prompt = global_prompt
+
+        self.current_timestep = environment.reset()
+        self.uuid = uuid.uuid4()  # Generate a unique id for the game
+        self.invalid_actions_retry = 5
+        self.args = args
+
+    @property
+    def num_players(self):
+        return self.environment.num_players
+
+    @property
+    def name_to_player(self) -> Dict[str, Player]:
+        return {player.name: player for player in self.players}
+
+    def reset(self) -> TimeStep:
+        # Reset the environment
+        self.current_timestep = self.environment.reset()
+        # Reset the players
+        for player in self.players:
+            player.reset()
+        # Reset the uuid
+        self.uuid = uuid.uuid4()
+        return self.current_timestep
+
+    def step(self) -> TimeStep:
+        """Take a step in the game: one player takes an action and the environment updates."""
+        player_name = self.environment.get_next_player()
+        player = self.name_to_player[player_name]  # get the player object
+        observation = self.environment.get_observation(
+            player_name
+        )  # get the observation for the player
+
+        timestep = None
+        for i in range(
+            self.invalid_actions_retry
+        ):  # try to take an action for a few times
+            action = player(observation)  # take an action
+            if self.environment.check_action(action, player_name):  # action is valid
+                timestep = self.environment.step(
+                    player_name, action
+                )  # update the environment
+                break
+            else:  # action is invalid
+                logging.warning(f"{player_name} made an invalid action {action}")
+                continue
+
+        if (
+            timestep is None
+        ):  # if the player made invalid actions for too many times, terminate the game
+            warning_msg = f"{player_name} has made invalid actions for {self.invalid_actions_retry} times. Terminating the game."
+            logging.warning(warning_msg)
+            raise TooManyInvalidActions(warning_msg)
+
+        return timestep
+
+    def next_is_human(self):
+        """Check if the next player is human."""
+        player_name = self.environment.get_next_player()
+        player = self.name_to_player[player_name]
+        return isinstance(player.backend, Human)
+
+    def run(self, num_steps: int = 1):
+        """Run the game for num_turns."""
+        for i in range(num_steps):
+            timestep = self.step()
+            if timestep.terminal:
+                break
+
+    @classmethod
+    def from_config(cls, config: Union[str, ArenaConfig]):
+        """Create an arena from a config."""
+        # If config is a path, load the config
+        if isinstance(config, str):
+            config = ArenaConfig.load(config)
+
+        global_prompt = config.get("global_prompt", None)
+
+        # Create the players
+        players = []
+        for player_config in config.players:
+            # Add public_prompt to the player config
+            if global_prompt is not None:
+                player_config["global_prompt"] = global_prompt
+
+            player = Player.from_config(player_config)
+            players.append(player)
+
+        # Check that the player names are unique
+        player_names = [player.name for player in players]
+        assert len(player_names) == len(
+            set(player_names)
+        ), "Player names must be unique"
+
+        # Create the environment
+        config.environment[
+            "player_names"
+        ] = player_names  # add the player names to the environment config
+        env = load_environment(config.environment)
+
+        return cls(players, env, global_prompt=global_prompt)
+
+    def to_config(self) -> ArenaConfig:
+        """Convert the arena to a config."""
+        # return {
+        #     "players": [player.to_config() for player in self.players],
+        #     "environment": self.environment.to_config(),
+        #     "global_prompt": self.global_prompt
+        # }
+        return ArenaConfig(
+            players=[player.to_config() for player in self.players],
+            environment=self.environment.to_config(),
+            global_prompt=self.global_prompt,
+        )
+
+    def launch_cli(self, max_steps: int = None, interactive: bool = True):
+        """Launch the command line interface."""
+        from agentreview.ui.cli import ArenaCLI
+
+        cli = ArenaCLI(self)
+        cli.launch(max_steps=max_steps, interactive=interactive)
+
+    def save_config(self, path: str):
+        """Save the config to a file."""
+        config = self.to_config()
+        config.save(path)
+
+    def save_history(self, path: str):
+        """
+        Save the history of the game to a file.
+
+        Supports csv and json formats.
+        """
+        messages = self.environment.get_observation()
+        message_rows = []
+
+        if path.endswith(".csv"):
+            header = [
+                "agent_name",
+                "content",
+                "turn",
+                "timestamp",
+                "visible_to",
+                "msg_type",
+            ]
+            for message in messages:
+                message_row = [
+                    message.agent_name,
+                    message.content,
+                    message.turn,
+                    str(message.timestamp),
+                    message.visible_to,
+                    message.msg_type,
+                ]
+                message_rows.append(message_row)
+
+            with open(path, "w") as f:
+                writer = csv.writer(f)
+                writer.writerow(header)
+                writer.writerows(message_rows)
+        elif path.endswith(".json"):
+            for message in messages:
+                message_row = {
+                    "agent_name": message.agent_name,
+                    "content": message.content,
+                    "turn": message.turn,
+                    "timestamp": str(message.timestamp),
+                    "visible_to": message.visible_to,
+                    "msg_type": message.msg_type,
+                }
+                message_rows.append(message_row)
+
+            with open(path, "w") as f:
+                json.dump(message_rows, f, indent=2)
+        else:
+            raise ValueError("Invalid file format")

agentreview/backends/__init__.py

@@ -0,0 +1,30 @@
+from ..config import BackendConfig
+from .anthropic import Claude
+from .base import IntelligenceBackend
+from .cohere import CohereAIChat
+from .hf_transformers import TransformersConversational
+from .human import Human
+from .openai import OpenAIChat
+from .dummy import Dummy
+
+ALL_BACKENDS = [
+    Human,
+    OpenAIChat,
+    CohereAIChat,
+    TransformersConversational,
+    Claude,
+    Dummy,
+]
+
+BACKEND_REGISTRY = {backend.type_name: backend for backend in ALL_BACKENDS}
+
+
+# Load a backend from a config dictionary
+def load_backend(config: BackendConfig):
+    try:
+        backend_cls = BACKEND_REGISTRY[config.backend_type]
+    except KeyError:
+        raise ValueError(f"Unknown backend type: {config.backend_type}")
+
+    backend = backend_cls.from_config(config)
+    return backend

agentreview/backends/anthropic.py

@@ -0,0 +1,119 @@
+import os
+import re
+from typing import List
+
+from tenacity import retry, stop_after_attempt, wait_random_exponential
+
+from ..message import SYSTEM_NAME as SYSTEM
+from ..message import Message
+from .base import IntelligenceBackend
+
+try:
+    import anthropic
+except ImportError:
+    is_anthropic_available = False
+    # logging.warning("anthropic package is not installed")
+else:
+    anthropic_api_key = os.environ.get("ANTHROPIC_API_KEY")
+    if anthropic_api_key is None:
+        # logging.warning("Anthropic API key is not set. Please set the environment variable ANTHROPIC_API_KEY")
+        is_anthropic_available = False
+    else:
+        is_anthropic_available = True
+
+DEFAULT_MAX_TOKENS = 256
+DEFAULT_MODEL = "claude-v1"
+
+
+class Claude(IntelligenceBackend):
+    """Interface to the Claude offered by Anthropic."""
+
+    stateful = False
+    type_name = "claude"
+
+    def __init__(
+        self, max_tokens: int = DEFAULT_MAX_TOKENS, model: str = DEFAULT_MODEL, **kwargs
+    ):
+        assert (
+            is_anthropic_available
+        ), "anthropic package is not installed or the API key is not set"
+        super().__init__(max_tokens=max_tokens, model=model, **kwargs)
+
+        self.max_tokens = max_tokens
+        self.model = model
+
+        self.client = anthropic.Client(os.environ["ANTHROPIC_API_KEY"])
+
+    @retry(stop=stop_after_attempt(6), wait=wait_random_exponential(min=1, max=60))
+    def _get_response(self, prompt: str):
+        response = self.client.completion(
+            prompt=prompt,
+            stop_sequences=[anthropic.HUMAN_PROMPT],
+            model=self.model,
+            max_tokens_to_sample=self.max_tokens,
+        )
+
+        response = response["completion"].strip()
+        return response
+
+    def query(
+        self,
+        agent_name: str,
+        role_desc: str,
+        history_messages: List[Message],
+        global_prompt: str = None,
+        request_msg: Message = None,
+        *args,
+        **kwargs,
+    ) -> str:
+        """
+        Format the input and call the Claude API.
+
+        args:
+            agent_name: the name of the agent
+            role_desc: the description of the role of the agent
+            env_desc: the description of the environment
+            history_messages: the history of the conversation, or the observation for the agent
+            request_msg: the request from the system to guide the agent's next response
+        """
+        all_messages = (
+            [(SYSTEM, global_prompt), (SYSTEM, role_desc)]
+            if global_prompt
+            else [(SYSTEM, role_desc)]
+        )
+
+        for message in history_messages:
+            all_messages.append((message.agent_name, message.content))
+        if request_msg:
+            all_messages.append((SYSTEM, request_msg.content))
+
+        prompt = ""
+        prev_is_human = False  # Whether the previous message is from human (in anthropic, the human is the user)
+        for i, message in enumerate(all_messages):
+            if i == 0:
+                assert (
+                    message[0] == SYSTEM
+                )  # The first message should be from the system
+
+            if message[0] == agent_name:
+                if prev_is_human:
+                    prompt = f"{prompt}{anthropic.AI_PROMPT} {message[1]}"
+                else:
+                    prompt = f"{prompt}\n\n{message[1]}"
+                prev_is_human = False
+            else:
+                if prev_is_human:
+                    prompt = f"{prompt}\n\n[{message[0]}]: {message[1]}"
+                else:
+                    prompt = f"{prompt}{anthropic.HUMAN_PROMPT}\n[{message[0]}]: {message[1]}"
+                prev_is_human = True
+        assert prev_is_human  # The last message should be from the human
+        # Add the AI prompt for Claude to generate the response
+        prompt = f"{prompt}{anthropic.AI_PROMPT}"
+
+        response = self._get_response(prompt, *args, **kwargs)
+
+        # Remove the agent name if the response starts with it
+        response = re.sub(rf"^\s*\[{agent_name}]:?", "", response).strip()
+
+        return response

agentreview/backends/bard.py

@@ -0,0 +1,90 @@
+import os
+import re
+from typing import List
+
+from tenacity import retry, stop_after_attempt, wait_random_exponential
+
+from ..message import SYSTEM_NAME as SYSTEM
+from ..message import Message
+from .base import IntelligenceBackend
+
+try:
+    import bardapi
+except ImportError:
+    is_bard_available = False
+    # logging.warning("bard package is not installed")
+else:
+    bard_api_key = os.environ.get("_BARD_API_KEY")
+    if bard_api_key is None:
+        # logging.warning(
+        #     "Bard API key is not set. Please set the environment variable _BARD_API_KEY")
+        is_bard_available = False
+    else:
+        is_bard_available = True
+
+DEFAULT_MAX_TOKENS = 4096
+
+
+class Bard(IntelligenceBackend):
+    """Interface to the Bard offered by Google."""
+
+    stateful = False
+    type_name = "bard"
+
+    def __init__(self, max_tokens: int = DEFAULT_MAX_TOKENS, **kwargs):
+        assert (
+            is_bard_available
+        ), "bard package is not installed or the API key is not set"
+        super().__init__(max_tokens=max_tokens, **kwargs)
+
+        self.max_tokens = max_tokens
+
+        self.client = bardapi.core.Bard()
+
+    @retry(stop=stop_after_attempt(6), wait=wait_random_exponential(min=1, max=60))
+    def _get_response(self, prompt: str):
+        response = self.client.get_answer(
+            input_text=prompt,
+        )
+
+        response = response["content"].strip()
+        return response
+
+    def query(
+        self,
+        agent_name: str,
+        role_desc: str,
+        history_messages: List[Message],
+        global_prompt: str = None,
+        request_msg: Message = None,
+        *args,
+        **kwargs,
+    ) -> str:
+        """
+        Format the input and call the Bard API.
+
+        args:
+            agent_name: the name of the agent
+            role_desc: the description of the role of the agent
+            env_desc: the description of the environment
+            history_messages: the history of the conversation, or the observation for the agent
+            request_msg: the request from the system to guide the agent's next response
+        """
+        all_messages = (
+            [(SYSTEM, global_prompt), (SYSTEM, role_desc)]
+            if global_prompt
+            else [(SYSTEM, role_desc)]
+        )
+
+        for message in history_messages:
+            all_messages.append((message.agent_name, message.content))
+        if request_msg:
+            all_messages.append((SYSTEM, request_msg.content))
+
+        # current bard api doesn't support role system, so just dump the raw messages as prompt
+        response = self._get_response(str(all_messages), *args, **kwargs)
+
+        # Remove the agent name if the response starts with it
+        response = re.sub(rf"^\s*\[{agent_name}]:?", "", response).strip()
+
+        return response

agentreview/backends/base.py

@@ -0,0 +1,66 @@
+from abc import abstractmethod
+from typing import List
+
+from ..config import BackendConfig, Configurable
+from ..message import Message
+
+
+class IntelligenceBackend(Configurable):
+    """An abstraction of the intelligence source of the agents."""
+
+    stateful = None
+    type_name = None
+
+    @abstractmethod
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)  # registers the arguments with Configurable
+
+    def __init_subclass__(cls, **kwargs):
+        # check if the subclass has the required attributes
+        for required in (
+            "stateful",
+            "type_name",
+        ):
+            if getattr(cls, required) is None:
+                raise TypeError(
+                    f"Can't instantiate abstract class {cls.__name__} without {required} attribute defined"
+                )
+        return super().__init_subclass__(**kwargs)
+
+    def to_config(self) -> BackendConfig:
+        self._config_dict["backend_type"] = self.type_name
+        return BackendConfig(**self._config_dict)
+
+    @abstractmethod
+    def query(
+        self,
+        agent_name: str,
+        role_desc: str,
+        history_messages: List[Message],
+        global_prompt: str = None,
+        request_msg: Message = None,
+        *args,
+        **kwargs,
+    ) -> str:
+        raise NotImplementedError
+
+    @abstractmethod
+    async def async_query(
+        self,
+        agent_name: str,
+        role_desc: str,
+        history_messages: List[Message],
+        global_prompt: str = None,
+        request_msg: Message = None,
+        *args,
+        **kwargs,
+    ) -> str:
+        """Async querying."""
+        raise NotImplementedError
+
+    # reset the state of the backend
+    def reset(self):
+        if self.stateful:
+            raise NotImplementedError
+        else:
+            pass

agentreview/backends/cohere.py

@@ -0,0 +1,126 @@
+import os
+from typing import List
+
+from tenacity import retry, stop_after_attempt, wait_random_exponential
+
+from ..message import Message
+from .base import IntelligenceBackend
+
+# Try to import the cohere package and check whether the API key is set
+try:
+    import cohere
+except ImportError:
+    is_cohere_available = False
+else:
+    if os.environ.get("COHEREAI_API_KEY") is None:
+        is_cohere_available = False
+    else:
+        is_cohere_available = True
+
+# Default config follows the [Cohere documentation](https://cohere-sdk.readthedocs.io/en/latest/cohere.html#cohere.client.Client.chat)
+DEFAULT_TEMPERATURE = 0.8
+DEFAULT_MAX_TOKENS = 200
+DEFAULT_MODEL = "command-xlarge"
+
+
+class CohereAIChat(IntelligenceBackend):
+    """Interface to the Cohere API."""
+
+    stateful = True
+    type_name = "cohere-chat"
+
+    def __init__(
+        self,
+        temperature: float = DEFAULT_TEMPERATURE,
+        max_tokens: int = DEFAULT_MAX_TOKENS,
+        model: str = DEFAULT_MODEL,
+        **kwargs,
+    ):
+        super().__init__(
+            temperature=temperature, max_tokens=max_tokens, model=model, **kwargs
+        )
+
+        self.temperature = temperature
+        self.max_tokens = max_tokens
+        self.model = model
+
+        assert (
+            is_cohere_available
+        ), "Cohere package is not installed or the API key is not set"
+        self.client = cohere.Client(os.environ.get("COHEREAI_API_KEY"))
+
+        # Stateful variables
+        self.session_id = None  # The session id for the last conversation
+        self.last_msg_hash = (
+            None  # The hash of the last message of the last conversation
+        )
+
+    def reset(self):
+        self.session_id = None
+        self.last_msg_hash = None
+
+    @retry(stop=stop_after_attempt(6), wait=wait_random_exponential(min=1, max=60))
+    def _get_response(self, new_message: str, persona_prompt: str):
+        response = self.client.chat(
+            new_message,
+            persona_prompt=persona_prompt,
+            temperature=self.temperature,
+            max_tokens=self.max_tokens,
+            session_id=self.session_id,
+        )
+
+        self.session_id = response.session_id  # Update the session id
+        return response.reply
+
+    def query(
+        self,
+        agent_name: str,
+        role_desc: str,
+        history_messages: List[Message],
+        global_prompt: str = None,
+        request_msg: Message = None,
+        *args,
+        **kwargs,
+    ) -> str:
+        """
+        Format the input and call the Cohere API.
+
+        args:
+            agent_name: the name of the agent
+            role_desc: the description of the role of the agent
+            env_desc: the description of the environment
+            history_messages: the history of the conversation, or the observation for the agent
+            request_msg: the request for the CohereAI
+        """
+        # Find the index of the last message of the last conversation
+        new_message_start_idx = 0
+        if self.last_msg_hash is not None:
+            for i, message in enumerate(history_messages):
+                if message.msg_hash == self.last_msg_hash:
+                    new_message_start_idx = i + 1
+                    break
+
+        new_messages = history_messages[new_message_start_idx:]
+        assert len(new_messages) > 0, "No new messages found (this should not happen)"
+
+        new_conversations = []
+        for message in new_messages:
+            if message.agent_name != agent_name:
+                # Since there are more than one player, we need to distinguish between the players
+                new_conversations.append(f"[{message.agent_name}]: {message.content}")
+
+        if request_msg:
+            new_conversations.append(
+                f"[{request_msg.agent_name}]: {request_msg.content}"
+            )
+
+        # Concatenate all new messages into one message because the Cohere API only accepts one message
+        new_message = "\n".join(new_conversations)
+        persona_prompt = f"Environment:\n{global_prompt}\n\nYour role:\n{role_desc}"
+
+        response = self._get_response(new_message, persona_prompt)
+
+        # Only update the last message hash if the API call is successful
+        self.last_msg_hash = new_messages[-1].msg_hash
+
+        return response

agentreview/backends/dummy.py

@@ -0,0 +1,14 @@
+from agentreview.config import Configurable
+
+
+class Dummy(Configurable):
+    """A dummy backend does not make any API calls. We use it for extracting paper contents in PaperExtractor
+    and also for testing."""
+    stateful = False
+    type_name = "dummy"
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+
+    def reset(self):
+        pass

agentreview/backends/hf_transformers.py

@@ -0,0 +1,127 @@
+import os
+from contextlib import contextmanager, redirect_stderr, redirect_stdout
+from typing import List
+
+from tenacity import retry, stop_after_attempt, wait_random_exponential
+
+from ..message import SYSTEM_NAME as SYSTEM
+from ..message import Message
+from .base import IntelligenceBackend
+
+
+@contextmanager
+def suppress_stdout_stderr():
+    """A context manager that redirects stdout and stderr to devnull."""
+    with open(os.devnull, "w") as fnull:
+        with redirect_stderr(fnull) as err, redirect_stdout(fnull) as out:
+            yield (err, out)
+
+
+with suppress_stdout_stderr():
+    # Try to import the transformers package
+    try:
+        import transformers
+        from transformers import pipeline
+        from transformers.pipelines.conversational import (
+            Conversation,
+            ConversationalPipeline,
+        )
+    except ImportError:
+        is_transformers_available = False
+    else:
+        is_transformers_available = True
+
+
+class TransformersConversational(IntelligenceBackend):
+    """Interface to the Transformers ConversationalPipeline."""
+
+    stateful = False
+    type_name = "transformers:conversational"
+
+    def __init__(self, model: str, device: int = -1, **kwargs):
+        super().__init__(model=model, device=device, **kwargs)
+        self.model = model
+        self.device = device
+
+        assert is_transformers_available, "Transformers package is not installed"
+        self.chatbot = pipeline(
+            task="conversational", model=self.model, device=self.device
+        )
+
+    @retry(stop=stop_after_attempt(6), wait=wait_random_exponential(min=1, max=60))
+    def _get_response(self, conversation):
+        conversation = self.chatbot(conversation)
+        response = conversation.generated_responses[-1]
+        return response
+
+    @staticmethod
+    def _msg_template(agent_name, content):
+        return f"[{agent_name}]: {content}"
+
+    def query(
+        self,
+        agent_name: str,
+        role_desc: str,
+        history_messages: List[Message],
+        global_prompt: str = None,
+        request_msg: Message = None,
+        *args,
+        **kwargs,
+    ) -> str:
+        user_inputs, generated_responses = [], []
+        all_messages = (
+            [(SYSTEM, global_prompt), (SYSTEM, role_desc)]
+            if global_prompt
+            else [(SYSTEM, role_desc)]
+        )
+
+        for msg in history_messages:
+            all_messages.append((msg.agent_name, msg.content))
+        if request_msg:
+            all_messages.append((SYSTEM, request_msg.content))
+
+        prev_is_user = False  # Whether the previous message is from the user
+        for i, message in enumerate(all_messages):
+            if i == 0:
+                assert (
+                    message[0] == SYSTEM
+                )  # The first message should be from the system
+
+            if message[0] != agent_name:
+                if not prev_is_user:
+                    user_inputs.append(self._msg_template(message[0], message[1]))
+                else:
+                    user_inputs[-1] += "\n" + self._msg_template(message[0], message[1])
+                prev_is_user = True
+            else:
+                if prev_is_user:
+                    generated_responses.append(message[1])
+                else:
+                    generated_responses[-1] += "\n" + message[1]
+                prev_is_user = False
+
+        assert len(user_inputs) == len(generated_responses) + 1
+        past_user_inputs = user_inputs[:-1]
+        new_user_input = user_inputs[-1]
+
+        # Recreate a conversation object from the history messages
+        conversation = Conversation(
+            text=new_user_input,
+            past_user_inputs=past_user_inputs,
+            generated_responses=generated_responses,
+        )
+
+        # Get the response
+        response = self._get_response(conversation)
+        return response
+
+
+# conversation = Conversation("Going to the movies tonight - any suggestions?")
+#
+# # Steps usually performed by the model when generating a response:
+# # 1. Mark the user input as processed (moved to the history)
+# conversation.mark_processed()
+# # 2. Append a mode response
+# conversation.append_response("The Big lebowski.")
+#
+# conversation.add_user_input("Is it good?")

agentreview/backends/human.py

@@ -0,0 +1,23 @@
+from ..config import BackendConfig
+from .base import IntelligenceBackend
+
+
+# An Error class for the human backend
+class HumanBackendError(Exception):
+    def __init__(self, agent_name: str):
+        self.agent_name = agent_name
+        super().__init__(f"Human backend requires a UI to get input from {agent_name}.")
+
+
+class Human(IntelligenceBackend):
+    stateful = False
+    type_name = "human"
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+
+    def to_config(self) -> BackendConfig:
+        return BackendConfig(backend_type=self.type_name)
+
+    def query(self, agent_name: str, **kwargs) -> str:
+        raise HumanBackendError(agent_name)

agentreview/backends/langchain.py

@@ -0,0 +1,169 @@
+import os
+import re
+from typing import List
+
+from tenacity import retry, stop_after_attempt, wait_random_exponential
+
+from ..message import SYSTEM_NAME, Message
+from .base import IntelligenceBackend
+
+try:
+    from langchain.llms import OpenAI
+except ImportError:
+    is_langchain_openai_available = False
+    # logging.warning("openai package is not installed")
+else:
+    api_key = os.environ.get("OPENAI_API_KEY")
+    if api_key is None:
+        # logging.warning("OpenAI API key is not set. Please set the environment variable OPENAI_API_KEY")
+        is_langchain_openai_available = False
+    else:
+        is_langchain_openai_available = True
+
+# Default config follows the OpenAI playground
+DEFAULT_TEMPERATURE = 0.7
+DEFAULT_MAX_TOKENS = 2048
+DEFAULT_MODEL = "gpt-4"
+
+END_OF_MESSAGE = "<EOS>"  # End of message token specified by us not OpenAI
+STOP = ("<|endoftext|>", END_OF_MESSAGE)  # End of sentence token
+BASE_PROMPT = f"The messages always end with the token {END_OF_MESSAGE}."
+
+
+class LangChainOpenAIChat(IntelligenceBackend):
+    """Interface to the ChatGPT style model with system, user, assistant roles separation."""
+
+    stateful = False
+    type_name = "openai-chat"
+
+    def __init__(
+        self,
+        temperature: float = DEFAULT_TEMPERATURE,
+        max_tokens: int = DEFAULT_MAX_TOKENS,
+        model: str = DEFAULT_MODEL,
+        merge_other_agents_as_one_user: bool = True,
+        **kwargs,
+    ):
+        """
+        Instantiate the OpenAIChat backend.
+
+        args:
+            temperature: the temperature of the sampling
+            max_tokens: the maximum number of tokens to sample
+            model: the model to use
+            merge_other_agents_as_one_user: whether to merge messages from other agents as one user message
+        """
+        assert (
+            is_langchain_openai_available
+        ), "langchain package is not installed or the API key is not set"
+        super().__init__(
+            temperature=temperature,
+            max_tokens=max_tokens,
+            model=model,
+            merge_other_agents_as_one_user=merge_other_agents_as_one_user,
+            **kwargs,
+        )
+
+        self.temperature = temperature
+        self.max_tokens = max_tokens
+        self.model = model
+        self.merge_other_agent_as_user = merge_other_agents_as_one_user
+        self.llm = OpenAI(
+            model_name=model,
+            temperature=temperature,
+            max_tokens=max_tokens,
+            openai_api_key=api_key,
+        )
+
+    @retry(stop=stop_after_attempt(6), wait=wait_random_exponential(min=1, max=60))
+    def _get_response(self, messages):
+        response = self.llm(prompt=messages, stop=STOP)
+        return response
+
+    def query(
+        self,
+        agent_name: str,
+        role_desc: str,
+        history_messages: List[Message],
+        global_prompt: str = None,
+        request_msg: Message = None,
+        *args,
+        **kwargs,
+    ) -> str:
+        """
+        Format the input and call the ChatGPT/GPT-4 API.
+
+        args:
+            agent_name: the name of the agent
+            role_desc: the description of the role of the agent
+            env_desc: the description of the environment
+            history_messages: the history of the conversation, or the observation for the agent
+            request_msg: the request from the system to guide the agent's next response
+        """
+
+        # Merge the role description and the global prompt as the system prompt for the agent
+        if global_prompt:  # Prepend the global prompt if it exists
+            system_prompt = f"{global_prompt.strip()}\n{BASE_PROMPT}\n\nYour name: {agent_name}\n\nYour role:{role_desc}"
+        else:
+            system_prompt = (
+                f"You are {agent_name}.\n\nYour role:{role_desc}\n\n{BASE_PROMPT}"
+            )
+
+        all_messages = [(SYSTEM_NAME, system_prompt)]
+        for msg in history_messages:
+            if msg.agent_name == SYSTEM_NAME:
+                all_messages.append((SYSTEM_NAME, msg.content))
+            else:  # non-system messages are suffixed with the end of message token
+                all_messages.append((msg.agent_name, f"{msg.content}{END_OF_MESSAGE}"))
+
+        if request_msg:
+            all_messages.append((SYSTEM_NAME, request_msg.content))
+        else:  # The default request message that reminds the agent its role and instruct it to speak
+            all_messages.append(
+                (SYSTEM_NAME, f"Now you speak, {agent_name}.{END_OF_MESSAGE}")
+            )
+
+        messages = []
+        for i, msg in enumerate(all_messages):
+            if i == 0:
+                assert (
+                    msg[0] == SYSTEM_NAME
+                )  # The first message should be from the system
+                messages.append({"role": "system", "content": msg[1]})
+            else:
+                if msg[0] == agent_name:
+                    messages.append({"role": "assistant", "content": msg[1]})
+                else:
+                    if messages[-1]["role"] == "user":  # last message is from user
+                        if self.merge_other_agent_as_user:
+                            messages[-1][
+                                "content"
+                            ] = f"{messages[-1]['content']}\n\n[{msg[0]}]: {msg[1]}"
+                        else:
+                            messages.append(
+                                {"role": "user", "content": f"[{msg[0]}]: {msg[1]}"}
+                            )
+                    elif (
+                        messages[-1]["role"] == "assistant"
+                    ):  # consecutive assistant messages
+                        # Merge the assistant messages
+                        messages[-1]["content"] = f"{messages[-1]['content']}\n{msg[1]}"
+                    elif messages[-1]["role"] == "system":
+                        messages.append(
+                            {"role": "user", "content": f"[{msg[0]}]: {msg[1]}"}
+                        )
+                    else:
+                        raise ValueError(f"Invalid role: {messages[-1]['role']}")
+
+        response = self._get_response(messages, *args, **kwargs)
+
+        # Remove the agent name if the response starts with it
+        response = re.sub(rf"^\s*\[.*]:", "", response).strip()  # noqa: F541
+        response = re.sub(
+            rf"^\s*{re.escape(agent_name)}\s*:", "", response
+        ).strip()  # noqa: F541
+
+        # Remove the tailing end of message token
+        response = re.sub(rf"{END_OF_MESSAGE}$", "", response).strip()
+
+        return response

agentreview/backends/openai.py

@@ -0,0 +1,180 @@
+import re
+from typing import List
+
+from tenacity import retry, stop_after_attempt, wait_random_exponential
+
+from arguments import parse_args
+from utility.authentication_utils import get_openai_client
+from .base import IntelligenceBackend
+from ..message import SYSTEM_NAME, Message
+
+args = parse_args()
+
+client = get_openai_client(client_type=args.openai_client_type)
+
+OPENAI_CLIENT_TYPE = args.openai_client_type
+
+# Default config follows the OpenAI playground
+DEFAULT_TEMPERATURE = 1.0
+DEFAULT_MAX_TOKENS = 4096
+
+# Check https://platform.openai.com/docs/models for more models
+
+DEFAULT_MODEL = "gpt-4o"
+
+END_OF_MESSAGE = "<EOS>"  # End of message token specified by us not OpenAI
+STOP = ("<|endoftext|>", END_OF_MESSAGE)  # End of sentence token
+BASE_PROMPT = f"The messages always end with the token {END_OF_MESSAGE}."
+
+
+class OpenAIChat(IntelligenceBackend):
+    """Interface to the ChatGPT style model with system, user, assistant roles separation."""
+
+    stateful = False
+    type_name = "openai-chat"
+
+    def __init__(
+            self,
+            temperature: float = DEFAULT_TEMPERATURE,
+            max_tokens: int = DEFAULT_MAX_TOKENS,
+            model: str = DEFAULT_MODEL,
+            merge_other_agents_as_one_user: bool = True,
+            **kwargs,
+    ):
+        """
+        Instantiate the OpenAIChat backend.
+
+        args:
+            temperature: the temperature of the sampling
+            max_tokens: the maximum number of tokens to sample
+            model: the model to use
+            merge_other_agents_as_one_user: whether to merge messages from other agents as one user message
+        """
+        super().__init__(
+            temperature=temperature,
+            max_tokens=max_tokens,
+            model=model,
+            merge_other_agents_as_one_user=merge_other_agents_as_one_user,
+            **kwargs,
+        )
+
+        self.temperature = temperature
+        self.max_tokens = max_tokens
+        self.model = model
+        self.merge_other_agent_as_user = merge_other_agents_as_one_user
+
+    @retry(stop=stop_after_attempt(6), wait=wait_random_exponential(min=1, max=60))
+    def _get_response(self, messages):
+        # Refer to https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/switching-endpoints for how to
+        # make API calls
+
+        if OPENAI_CLIENT_TYPE == "openai":
+            completion = client.chat.completions.create(
+                model=self.model,
+                messages=messages,
+                temperature=self.temperature,
+                max_tokens=self.max_tokens,
+                stop=STOP,
+            )
+
+        elif OPENAI_CLIENT_TYPE == "azure_openai":
+            completion = client.chat.completions.create(
+                model=self.model,
+                messages=messages,
+                temperature=self.temperature,
+                max_tokens=self.max_tokens,
+                stop=STOP,
+            )
+
+        else:
+            raise NotImplementedError
+
+        response = completion.choices[0].message.content
+        response = response.strip()
+        return response
+
+    def query(
+            self,
+            agent_name: str,
+            role_desc: str,
+            history_messages: List[Message],
+            global_prompt: str = None,
+            request_msg: Message = None,
+            *args,
+            **kwargs,
+    ) -> str:
+        """
+        Format the input and call the ChatGPT/GPT-4 API.
+
+        args:
+            agent_name: the name of the agent
+            role_desc: the description of the role of the agent
+            env_desc: the description of the environment
+            history_messages: the history of the conversation, or the observation for the agent
+            request_msg: the request from the system to guide the agent's next response
+        """
+
+        # Merge the role description and the global prompt as the system prompt for the agent
+        if global_prompt:  # Prepend the global prompt if it exists
+            system_prompt = f"You are a helpful assistant.\n{global_prompt.strip()}\n{BASE_PROMPT}\n\nYour name is {agent_name}.\n\nYour role:{role_desc}"
+        else:
+            system_prompt = f"You are a helpful assistant. Your name is {agent_name}.\n\nYour role:{role_desc}\n\n{BASE_PROMPT}"
+
+        all_messages = [(SYSTEM_NAME, system_prompt)]
+        for msg in history_messages:
+            if msg.agent_name == SYSTEM_NAME:
+                all_messages.append((SYSTEM_NAME, msg.content))
+            else:  # non-system messages are suffixed with the end of message token
+                all_messages.append((msg.agent_name, f"{msg.content}{END_OF_MESSAGE}"))
+
+        if request_msg:
+            all_messages.append((SYSTEM_NAME, request_msg.content))
+        else:  # The default request message that reminds the agent its role and instruct it to speak
+            all_messages.append(
+                (SYSTEM_NAME, f"Now you speak, {agent_name}.{END_OF_MESSAGE}")
+            )
+
+        messages = []
+        for i, msg in enumerate(all_messages):
+            if i == 0:
+                assert (
+                        msg[0] == SYSTEM_NAME
+                )  # The first message should be from the system
+                messages.append({"role": "system", "content": msg[1]})
+            else:
+                if msg[0] == agent_name:
+                    messages.append({"role": "assistant", "content": msg[1]})
+                else:
+                    if messages[-1]["role"] == "user":  # last message is from user
+                        if self.merge_other_agent_as_user:
+                            messages[-1][
+                                "content"
+                            ] = f"{messages[-1]['content']}\n\n[{msg[0]}]: {msg[1]}"
+                        else:
+                            messages.append(
+                                {"role": "user", "content": f"[{msg[0]}]: {msg[1]}"}
+                            )
+                    elif (
+                            messages[-1]["role"] == "assistant"
+                    ):  # consecutive assistant messages
+                        # Merge the assistant messages
+                        messages[-1]["content"] = f"{messages[-1]['content']}\n{msg[1]}"
+                    elif messages[-1]["role"] == "system":
+                        messages.append(
+                            {"role": "user", "content": f"[{msg[0]}]: {msg[1]}"}
+                        )
+                    else:
+                        raise ValueError(f"Invalid role: {messages[-1]['role']}")
+
+        response = self._get_response(messages, *args, **kwargs)
+
+        # Remove the agent name if the response starts with it
+        response = re.sub(rf"^\s*\[.*]:", "", response).strip()  # noqa: F541
+        response = re.sub(
+            rf"^\s*{re.escape(agent_name)}\s*:", "", response
+        ).strip()  # noqa: F451
+
+        # Remove the tailing end of message token
+        response = re.sub(rf"{END_OF_MESSAGE}$", "", response).strip()
+
+        return response

agentreview/config.py

@@ -0,0 +1,143 @@
+import copy
+import json
+
+from .utils import AttributedDict
+
+
+class Config(AttributedDict):
+    """
+    Config class to manage the configuration of the games.
+
+    The class has a few useful methods to load and save the config.
+    """
+
+    # convert dict to Config recursively
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        for key, value in self.items():
+
+            # Try to convert the value (the "metadata" field) to dict if applicable
+            try:
+                value = dict(eval(value))
+            except Exception:
+                pass
+
+            if isinstance(value, dict):
+                self[key] = init_config(value)  # convert dict to Config recursively
+            # convert list of dict to list of Config recursively
+            elif isinstance(value, list) and len(value) > 0:
+                self[key] = [
+                    init_config(item) if isinstance(item, dict) else item
+                    for item in value
+                ]
+
+    def save(self, path: str):
+        # save config to file
+        with open(path, "w") as f:
+            json.dump(self, f, indent=4)
+
+    @classmethod
+    def load(cls, path: str):
+        # load config from file
+        with open(path) as f:
+            config = json.load(f)
+        return cls(config)
+
+    def deepcopy(self):
+        # get the config class so that subclasses can be copied in the correct class
+        config_class = self.__class__
+        # make a deep copy of the config
+        return config_class(copy.deepcopy(self))
+
+
+class Configurable:
+    """Configurable is an interface for classes that can be initialized with a config."""
+
+    def __init__(self, **kwargs):
+        self._config_dict = kwargs
+
+    @classmethod
+    def from_config(cls, config: Config):
+        return cls(**config)
+
+    def to_config(self) -> Config:
+        # Convert the _config_dict to Config
+        return Config(**self._config_dict)
+
+    def save_config(self, path: str):
+        self.to_config().save(path)
+
+
+class EnvironmentConfig(Config):
+    """EnvironmentConfig contains a env_type field to indicate the name of the environment."""
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        # check if the env_type field is specified
+        if "env_type" not in self:
+            raise ValueError("The env_type field is not specified")
+
+
+class BackendConfig(Config):
+    """BackendConfig contains a backend_type field to indicate the name of the backend."""
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        # check if the backend_type field is specified
+        if "backend_type" not in self:
+            raise ValueError("The backend_type field is not specified")
+
+
+class AgentConfig(Config):
+    """AgentConfig contains role_desc and backend fields."""
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        # check if the role_desc field is specified
+        if "role_desc" not in self:
+            raise ValueError("The role_desc field is not specified")
+        # check if the backend field is specified
+        if "backend" not in self:
+            raise ValueError("The backend field is not specified")
+        # Make sure the backend field is a BackendConfig
+        if not isinstance(self["backend"], BackendConfig):
+            raise ValueError("The backend field must be a BackendConfig")
+
+
+class ArenaConfig(Config):
+    """ArenaConfig contains a list of AgentConfig."""
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        # check if the players field is specified and it is List[AgentConfig]
+        if "players" not in self:
+            raise ValueError("The players field is not specified")
+        if not isinstance(self["players"], list):
+            raise ValueError("The players field must be a list")
+        for player in self["players"]:
+            if not isinstance(player, AgentConfig):
+                raise ValueError("The players field must be a list of AgentConfig")
+
+        # check if environment field is specified and it is EnvironmentConfig
+        if "environment" not in self:
+            raise ValueError("The environment field is not specified")
+        if not isinstance(self["environment"], EnvironmentConfig):
+            raise ValueError("The environment field must be an EnvironmentConfig")
+
+
+# Initialize with different config class depending on whether the config is for environment or backend
+def init_config(config: dict):
+    if not isinstance(config, dict):
+        raise ValueError("The config must be a dict")
+
+    # check if the config is for environment or backend
+    if "env_type" in config:
+        return EnvironmentConfig(config)
+    elif "backend_type" in config:
+        return BackendConfig(config)
+    elif "role_desc" in config:
+        return AgentConfig(config)
+    elif "players" in config:
+        return ArenaConfig(config)
+    else:
+        return Config(config)

agentreview/database.py

@@ -0,0 +1,136 @@
+"""
+Datastore module for chat_arena.
+
+This module provides utilities for storing the messages and the game results into database.
+Currently, it supports Supabase.
+"""
+import json
+import os
+import uuid
+from typing import List
+
+from .arena import Arena
+from .message import Message
+
+# Attempt importing Supabase
+try:
+    import supabase
+
+    # Get the Supabase URL and secret key from environment variables
+    SUPABASE_URL = os.environ.get("SUPABASE_URL", "")
+    SUPABASE_SECRET_KEY = os.environ.get("SUPABASE_SECRET_KEY", "")
+    assert SUPABASE_URL and SUPABASE_SECRET_KEY
+except Exception:
+    supabase_available = False
+else:
+    supabase_available = True
+
+
+# Store the messages into the Supabase database
+class SupabaseDB:
+    def __init__(self):
+        assert supabase_available and SUPABASE_URL and SUPABASE_SECRET_KEY
+        supabase_client = supabase.create_client(SUPABASE_URL, SUPABASE_SECRET_KEY)
+        self.client = supabase_client
+
+    # Save Arena state to Supabase
+    def save_arena(self, arena: Arena):
+        # Save the environment config
+        self._save_environment(arena)
+
+        # Save the player configs
+        self._save_player_configs(arena)
+
+        # Save the messages
+        self.save_messages(arena)
+
+    # Save the environment config of the arena
+    def _save_environment(self, arena: Arena):
+        env = arena.environment
+        env_config = env.to_config()
+        moderator_config = env_config.pop("moderator", None)
+
+        arena_row = {
+            "arena_id": str(arena.uuid),
+            "global_prompt": arena.global_prompt,
+            "env_type": env_config["env_type"],
+            "env_config": json.dumps(env_config),
+        }
+        self.client.table("Arena").insert(arena_row).execute()
+
+        # Get the moderator config
+        if moderator_config:
+            moderator_row = {
+                "moderator_id": str(
+                    uuid.uuid5(arena.uuid, json.dumps(moderator_config))
+                ),
+                "arena_id": str(arena.uuid),
+                "role_desc": moderator_config["role_desc"],
+                "terminal_condition": moderator_config["terminal_condition"],
+                "backend_type": moderator_config["backend"]["backend_type"],
+                "temperature": moderator_config["backend"]["temperature"],
+                "max_tokens": moderator_config["backend"]["max_tokens"],
+            }
+            self.client.table("Moderator").insert(moderator_row).execute()
+
+    # Save the player configs of the arena
+    def _save_player_configs(self, arena: Arena):
+        player_rows = []
+        for player in arena.players:
+            player_config = player.to_config()
+            player_row = {
+                "player_id": str(uuid.uuid5(arena.uuid, json.dumps(player_config))),
+                "arena_id": str(arena.uuid),
+                "name": player.name,
+                "role_desc": player_config["role_desc"],
+                "backend_type": player_config["backend"]["backend_type"],
+                "temperature": player_config["backend"].get("temperature", None),
+                "max_tokens": player_config["backend"].get("max_tokens", None),
+            }
+            player_rows.append(player_row)
+
+        self.client.table("Player").insert(player_rows).execute()
+
+    # Save the messages
+    def save_messages(self, arena: Arena, messages: List[Message] = None):
+        if messages is None:
+            messages = arena.environment.get_observation()
+
+        # Filter messages that are already logged
+        messages = [msg for msg in messages if not msg.logged]
+
+        message_rows = []
+        for message in messages:
+            message_row = {
+                "message_id": str(uuid.uuid5(arena.uuid, message.msg_hash)),
+                "arena_id": str(arena.uuid),
+                "agent_name": message.agent_name,
+                "content": message.content,
+                "turn": message.turn,
+                "timestamp": str(message.timestamp),
+                "msg_type": message.msg_type,
+                "visible_to": json.dumps(message.visible_to),
+            }
+            message_rows.append(message_row)
+
+        self.client.table("Message").insert(message_rows).execute()
+
+        # Mark the messages as logged
+        for message in messages:
+            message.logged = True
+
+
+# Log the arena results into the Supabase database
+def log_arena(arena: Arena, database=None):
+    if database is None:
+        pass
+    else:
+        database.save_arena(arena)
+
+
+# Log the messages into the Supabase database
+def log_messages(arena: Arena, messages: List[Message], database=None):
+    if database is None:
+        pass
+    else:
+        database.save_messages(arena, messages)

agentreview/dataset/download_openreview_paper.py

@@ -0,0 +1,136 @@
+"""
+Download all papers from one year of ICLR conference using OpenReview API.
+
+This script downloads all paper PDFs and their corresponding metadata
+from the ICLR 2023 conference using the OpenReview API.
+
+Alternative methods to download can be found in this
+[colab notebook](https://colab.research.google.com/drive/1vXXNxn8lnO3j1dgoidjybbKIN0DW0Bt2),
+though it's not used here.
+"""
+
+import glob
+import json
+import os
+import time
+import requests
+
+from arguments import parse_args
+
+try:
+    import openreview
+except ImportError:
+    raise ImportError("Please install openreview package using `pip install openreview-py`")
+
+def download_papers():
+    """Downloads all papers from ICLR 2023 using OpenReview API.
+
+    This function authenticates with the OpenReview API using environment
+    variables for the username and password. It then iterates through the
+    available papers, downloads the PDF, and saves the corresponding metadata
+    (in JSON format) in the specified directories.
+
+    Raises:
+        AssertionError: If the OPENREVIEW_USERNAME or OPENREVIEW_PASSWORD environment
+                        variables are not set.
+        AssertionError: If the conference argument is not for ICLR.
+    """
+
+    args = parse_args()
+
+    openreview_username = os.environ.get("OPENREVIEW_USERNAME")
+    openreview_password = os.environ.get("OPENREVIEW_PASSWORD")
+
+    assert openreview_username is not None, (
+        "Please set your OpenReview username through the OPENREVIEW_USERNAME environment variable."
+    )
+    assert openreview_password is not None, (
+        "Please set your OpenReview password through the OPENREVIEW_PASSWORD environment variable."
+    )
+
+    client = openreview.Client(
+        baseurl='https://api.openreview.net',
+        username=openreview_username,
+        password=openreview_password
+    )
+
+    page_size = 1000
+    offset = 0
+    papers_directory = os.path.join(args.data_dir, args.conference, "paper")
+    notes_directory = os.path.join(args.data_dir, args.conference, "notes")
+
+    assert "ICLR" in args.conference, "Only works for ICLR conferences!"
+    year = int(args.conference.split("ICLR")[-1])  # Only works for ICLR currently
+    ids = []
+
+    # Create directories if they don't exist
+    for path in [papers_directory, notes_directory]:
+        os.makedirs(path, exist_ok=True)
+
+    while True:
+        # Fetch submissions with pagination
+        notes = client.get_notes(
+            invitation=f'ICLR.cc/{year}/Conference/-/Blind_Submission',
+            details='all',
+            offset=offset,
+            limit=page_size
+        )
+
+        if not notes:
+            break  # Exit if no more notes are available
+
+        # Get existing paper IDs to avoid re-downloading
+        existing_papers = glob.glob(f"{papers_directory}/*.pdf")
+        existing_paper_ids = {int(os.path.basename(paper).split(".pdf")[0]) for paper in existing_papers}
+
+        for note in notes:
+            paper_id = note.number
+            paper_path = os.path.join(papers_directory, f"{paper_id}.pdf")
+            note_path = os.path.join(notes_directory, f"{paper_id}.json")
+
+            # Skip existing papers
+            if paper_id in existing_paper_ids:
+                print(f"Paper {paper_id} already downloaded.")
+                continue
+
+            print(f"Title: {note.content.get('title', 'N/A')}")
+            print(f"Abstract: {note.content.get('abstract', 'N/A')}")
+            print(f"TL;DR: {note.content.get('TL;DR', 'N/A')}")
+            pdf_link = f"https://openreview.net/pdf?id={note.id}"
+            print(f"PDF Link: {pdf_link}")
+
+            # Attempt to download the paper PDF, retry if fails
+            tries = 0
+            while tries < 10:
+                try:
+                    response = requests.get(pdf_link)
+
+                    if response.status_code == 200:
+                        
+                        with open(paper_path, "wb") as pdf_file:
+                            pdf_file.write(response.content)
+
+                        print(f"PDF downloaded successfully as {paper_path}")
+
+                        # Save metadata as JSON, which contains the reviews, rebuttals, and decisions. 
+                        with open(note_path, "w") as note_file:
+                            json.dump(note.to_json(), note_file, indent=2)
+
+                        break
+
+                    else:
+                        print(f"Attempt {tries} failed. Status code: {response.status_code}")
+                        if response.status_code == 429:  # Too many requests
+                            print("Too many requests. Sleeping for 10 seconds.")
+                            time.sleep(10)
+
+                except Exception as e:
+                    print(f"Attempt {tries} failed with error: {e}")
+
+                tries += 1
+
+        offset += page_size
+
+
+if __name__ == "__main__":
+    download_papers()

agentreview/dataset/process_submissions.py

@@ -0,0 +1,113 @@
+"""
+Process and classify ICLR submissions using OpenReview API.
+
+This script processes ICLR submissions, classifies them into subdirectories
+based on decisions, extracts paper content into JSON format, and checks the
+validity of the processed papers.
+
+It includes three main functions:
+- classify_ICLR_submissions_into_subdirectories: Classifies papers into
+  directories based on decisions.
+- process_submission: Processes each submission by extracting text and saving
+  it as a JSON file.
+- check_processed_paper: Verifies if all processed papers are valid JSON files.
+"""
+
+import os
+import sys
+import traceback
+from collections import Counter
+
+from tqdm import tqdm
+
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
+
+import const
+from arguments import parse_args
+from utility.utils import print_colored
+
+decision_map = {
+    # ICLR 2023
+    "Reject": "Reject",
+    "Accept: poster": "Accept-poster",
+    "Accept: notable-top-25%": "Accept-notable-top-25",
+    "Accept: notable-top-5%": "Accept-notable-top-5",
+
+    # ICLR 2022
+    "Accept (Poster)": "Accept-poster",
+    "Accept (Oral)": "Accept-oral",
+    "Accept (Spotlight)": "Accept-spotlight",
+
+    # ICLR 2021
+    "Significant concerns (Do not publish)": "Significant-concerns",
+    "Concerns raised (can publish with adjustment)": "Concerns-raised",
+
+    # ICLR 2020
+    "Accept (Talk)": "Accept-oral",  # We assume this signifies an oral presentation
+
+    # ICLR 2018
+    "Invite to Workshop Track": "Reject"
+}
+
+
+def categorize_ICLR_submissions_into_subdirectories():
+    """Classifies ICLR submissions into subdirectories based on review decisions.
+
+    This function iterates through the review notes and identifies the decision
+    (recommendation or final decision) for each submission. It then moves the
+    notes and their corresponding papers into directories based on the decision.
+
+    Raises:
+        AssertionError: If the line containing the decision does not have the
+                        expected format.
+    """
+    note_dir = f"data/{args.conference}/notes"
+    paper_dir = f"data/{args.conference}/paper"
+
+    for note in os.listdir(note_dir):
+        print(note)
+
+        # Skip directories or irrelevant files
+        if os.path.isdir(os.path.join(note_dir, note)) or ".DS_Store" in note:
+            continue
+
+        note_path = os.path.join(note_dir, note)
+        lines = open(note_path, "r").readlines()
+        decision = None
+
+        for line in tqdm(lines):
+            if "\"recommendation\"" in line:
+                assert Counter(line)["\""] == 4, "Unexpected format in recommendation line."
+                print(line)
+                decision = line.split("\"recommendation\"")[1].split("\"")[1]
+                break
+
+            elif "\"decision\"" in line:
+                assert Counter(line)["\""] == 4, "Unexpected format in decision line."
+                print(line)
+                try:
+                    decision = line.split("\"decision\"")[1].split("\"")[1]
+                    break
+                except Exception:
+                    traceback.print_exc()
+                    print_colored(line, 'red')
+
+        if decision is None:
+            # Possibly withdrawn papers
+            print_colored(f"Could not find decision for {note}", "red")
+            continue
+
+        os.makedirs(os.path.join(note_dir, decision_map[decision]), exist_ok=True)
+        os.makedirs(os.path.join(paper_dir, decision_map[decision]), exist_ok=True)
+        os.rename(note_path, os.path.join(note_dir, decision_map[decision], note))
+
+        paper_id = int(note.split(".json")[0])
+        paper_path = os.path.join(paper_dir, f"{paper_id}.pdf")
+        os.rename(paper_path, os.path.join(paper_dir, decision_map[decision], f"{paper_id}.pdf"))
+
+
+if __name__ == "__main__":
+    args = parse_args()
+
+    # Extract contents of each paper into a JSON file
+    categorize_ICLR_submissions_into_subdirectories()

agentreview/environments/__init__.py

@@ -0,0 +1,25 @@
+from ..config import EnvironmentConfig
+from .base import Environment, TimeStep
+from .conversation import Conversation, ModeratedConversation
+from .paper_review import PaperReview
+from .paper_decision import PaperDecision
+
+ALL_ENVIRONMENTS = [
+    Conversation,
+    ModeratedConversation,
+    PaperReview,
+    PaperDecision,
+]
+
+ENV_REGISTRY = {env.type_name: env for env in ALL_ENVIRONMENTS}
+
+
+# Load an environment from a config dictionary
+def load_environment(config: EnvironmentConfig):
+    try:
+        env_cls = ENV_REGISTRY[config["env_type"]]
+    except KeyError:
+        raise ValueError(f"Unknown environment type: {config['env_type']}")
+
+    env = env_cls.from_config(config)
+    return env

agentreview/environments/base.py

@@ -0,0 +1,188 @@
+from abc import abstractmethod
+from dataclasses import dataclass
+from typing import Dict, List
+
+from ..config import Configurable, EnvironmentConfig
+from ..message import Message
+from ..utils import AttributedDict
+
+
+@dataclass
+class TimeStep(AttributedDict):
+    """
+    Represents a single step in time within the simulation.
+
+    It includes observation, reward, and terminal state.
+
+    Attributes:
+        observation (List[Message]): A list of messages (observations) for the current timestep.
+        reward (Dict[str, float]): A dictionary with player names as keys and corresponding rewards as values.
+        terminal (bool): A boolean indicating whether the current state is terminal (end of episode).
+    """
+
+    observation: List[Message]
+    reward: Dict[str, float]
+    terminal: bool
+
+
+class Environment(Configurable):
+    """
+    Abstract class representing an environment.
+
+    It defines the necessary methods any environment must implement.
+
+    Inherits from:
+        Configurable: A custom class that provides methods to handle configuration settings.
+
+    Attributes:
+        type_name (str): Type of the environment, typically set to the lower case of the class name.
+
+    Note:
+        Subclasses should override and implement the abstract methods defined here.
+    """
+
+    type_name = None
+    phase_index = 0
+    task = None
+    @abstractmethod
+    def __init__(self, player_names: List[str], **kwargs):
+        """
+        Initialize the Environment.
+
+        Parameters:
+            player_names (List[str]): Names of the players in the environment.
+        """
+        super().__init__(
+            player_names=player_names, **kwargs
+        )  # registers the arguments with Configurable
+        self.player_names = player_names
+
+    def __init_subclass__(cls, **kwargs):
+        """
+        Automatically called when a subclass is being initialized.
+
+        Here it's used to check if the subclass has the required attributes.
+        """
+        for required in ("type_name",):
+            if getattr(cls, required) is None:
+                cls.type_name = cls.__name__.lower()
+
+        return super().__init_subclass__(**kwargs)
+
+    @abstractmethod
+    def reset(self):
+        """
+        Reset the environment to its initial state.
+
+        Note:
+            This method must be implemented by subclasses.
+        """
+        pass
+
+    def to_config(self) -> EnvironmentConfig:
+        self._config_dict["env_type"] = self.type_name
+        return EnvironmentConfig(**self._config_dict)
+
+    @property
+    def num_players(self) -> int:
+        """Get the number of players."""
+        return len(self.player_names)
+
+    @abstractmethod
+    def get_next_player(self) -> str:
+        """
+        Return the name of the next player.
+
+        Note:
+            This method must be implemented by subclasses.
+
+        Returns:
+            str: The name of the next player.
+        """
+        pass
+
+    @abstractmethod
+    def get_observation(self, player_name=None) -> List[Message]:
+        """
+        Return observation for a given player.
+
+        Note:
+            This method must be implemented by subclasses.
+
+        Parameters:
+            player_name (str, optional): The name of the player for whom to get the observation.
+
+        Returns:
+            List[Message]: The observation for the player in the form of a list of messages.
+        """
+        pass
+
+    @abstractmethod
+    def print(self):
+        """Print the environment state."""
+        pass
+
+    @abstractmethod
+    def step(self, player_name: str, action: str) -> TimeStep:
+        """
+        Execute a step in the environment given an action from a player.
+
+        Note:
+            This method must be implemented by subclasses.
+
+        Parameters:
+            player_name (str): The name of the player.
+            action (str): The action that the player wants to take.
+
+        Returns:
+            TimeStep: An object of the TimeStep class containing the observation, reward, and done state.
+        """
+        pass
+
+    @abstractmethod
+    def check_action(self, action: str, player_name: str) -> bool:
+        """
+        Check whether a given action is valid for a player.
+
+        Note:
+            This method must be implemented by subclasses.
+
+        Parameters:
+            action (str): The action to be checked.
+            player_name (str): The name of the player.
+
+        Returns:
+            bool: True if the action is valid, False otherwise.
+        """
+        return True
+
+    @abstractmethod
+    def is_terminal(self) -> bool:
+        """
+        Check whether the environment is in a terminal state (end of episode).
+
+        Note:
+            This method must be implemented by subclasses.
+
+        Returns:
+            bool: True if the environment is in a terminal state, False otherwise.
+        """
+        pass
+
+    def get_zero_rewards(self) -> Dict[str, float]:
+        """
+        Return a dictionary with all player names as keys and zero as reward.
+
+        Returns:
+            Dict[str, float]: A dictionary of players and their rewards (all zero).
+        """
+        return {player_name: 0.0 for player_name in self.player_names}
+
+    def get_one_rewards(self) -> Dict[str, float]:
+        """
+        Return a dictionary with all player names as keys and one as reward.
+
+        Returns:
+            Dict[str, float]: A dictionary of players and their rewards (all one).
+        """
+        return {player_name: 1.0 for player_name in self.player_names}

agentreview/environments/conversation.py

@@ -0,0 +1,198 @@
+from typing import List, Union
+
+from ..agent import SIGNAL_END_OF_CONVERSATION, Moderator
+from ..config import AgentConfig, EnvironmentConfig
+from ..message import Message, MessagePool
+from .base import Environment, TimeStep
+
+
+class Conversation(Environment):
+    """
+    Turn-based fully observable conversation environment.
+
+    Next speaker order is either parallel or round-robin.
+    """
+
+    type_name = "conversation"
+
+    def __init__(self, player_names: List[str], parallel: bool = False, **kwargs):
+        super().__init__(player_names=player_names, parallel=parallel, **kwargs)
+
+        self.parallel = parallel
+
+        # The "state" of the environment is maintained by the message pool
+        self.message_pool = MessagePool()
+
+        self._current_turn = 0
+        self._next_player_index = 0
+
+    def reset(self):
+        self._current_turn = 0
+        self._next_player_index = 0
+        self.message_pool.reset()
+
+        init_timestep = TimeStep(
+            observation=[], reward=self.get_zero_rewards(), terminal=False
+        )
+        return init_timestep
+
+    @property
+    def phase_index(self):
+        return self._phase_index
+
+    @phase_index.setter
+    def phase_index(self, value):
+        self._phase_index = value
+
+    def to_config(self) -> EnvironmentConfig:
+        return EnvironmentConfig(
+            env_type=self.type_name,
+            player_names=self.player_names,
+            parallel=self.parallel,
+        )
+
+    def print(self):
+        self.message_pool.print()
+
+    def get_next_player(self) -> str:
+        """Get the next player."""
+        return self.player_names[self._next_player_index]
+
+    def get_observation(self, player_name=None) -> List[Message]:
+        """Get observation for the player."""
+        if player_name is None:
+            return self.message_pool.get_all_messages()
+        else:
+            return self.message_pool.get_visible_messages(
+                player_name, turn=self._current_turn
+            )
+
+    def is_terminal(self) -> bool:
+        """Check if the conversation is over."""
+        # If the last message is the signal, then the conversation is over
+        if self.message_pool.last_message.content.startswith(
+            SIGNAL_END_OF_CONVERSATION
+        ):
+            return True
+
+    def step(self, player_name: str, action: str) -> TimeStep:
+        """
+        Step function that is called by the arena.
+
+        Args:
+            player_name: the name of the player that takes the action
+            action: the action that the agents wants to take
+        """
+        message = Message(
+            agent_name=player_name, content=action, turn=self._current_turn
+        )
+        self.message_pool.append_message(message)
+
+        # Update the counters
+        if not self.parallel or self._next_player_index == 0:
+            self._current_turn += 1
+        self._next_player_index = (self._next_player_index + 1) % self.num_players
+
+        timestep = TimeStep(
+            observation=self.get_observation(),
+            reward=self.get_zero_rewards(),
+            terminal=self.is_terminal(),
+        )  # Return all the messages
+        return timestep
+
+
+class ModeratedConversation(Conversation):
+    """
+    Turn-based fully observable conversation environment.
+
+    Next speaker order is either parallel or round-robin.
+    Moderator is a special agent that can see all messages and can decide whether the conversation is over.
+    """
+
+    type_name = "moderated_conversation"
+
+    def __init__(
+        self,
+        player_names: List[str],
+        moderator: Union[Moderator, AgentConfig],
+        parallel: bool = False,
+        moderator_visibility="all",
+        moderator_period=None,
+        **kwargs,
+    ):
+        super().__init__(player_names=player_names, parallel=parallel, **kwargs)
+
+        if isinstance(moderator, AgentConfig):
+            moderator_config = moderator
+            moderator = Moderator.from_config(moderator_config)
+        elif not isinstance(moderator, Moderator):
+            raise ValueError(
+                "moderator must be either an AgentConfig or a Moderator instance."
+            )
+
+        self.moderator = moderator
+        self.moderator_visibility = moderator_visibility
+        if moderator_period is None:
+            if parallel:
+                self.moderator_period = "round"
+            else:
+                self.moderator_period = "turn"
+        else:
+            self.moderator_period = moderator_period
+
+    def to_config(self) -> EnvironmentConfig:
+        # This environment contains some special config arguments that needs to be handle specially
+        return EnvironmentConfig(
+            env_type=self.type_name,
+            player_names=self.player_names,
+            parallel=self.parallel,
+            moderator=self.moderator.to_config(),
+            moderator_visibility=self.moderator_visibility,
+            moderator_period=self.moderator_period,
+        )
+
+    def step(self, player_name: str, action: str) -> TimeStep:
+        """
+        Step function that is called by the arena.
+
+        Args:
+            player_name: the name of the player that takes the action
+            action: the action that the agents wants to take
+        """
+        message = Message(
+            agent_name=player_name, content=action, turn=self._current_turn
+        )
+        self.message_pool.append_message(message)
+
+        # Round-robin order for the next player
+        self._next_player_index = (self._next_player_index + 1) % self.num_players
+
+        if self.moderator_period == "turn" or (
+            self.moderator_period == "round" and self._next_player_index == 0
+        ):
+            # Moderator's turn
+            moderator_history = self.message_pool.get_all_messages()
+            moderator_response = self.moderator(moderator_history)
+            moderator_message = Message(
+                agent_name=self.moderator.name,
+                content=moderator_response,
+                turn=self._current_turn,
+                visible_to=self.moderator_visibility,
+            )
+            self.message_pool.append_message(moderator_message)
+            terminal = (
+                self.moderator.is_terminal(moderator_history) or self.is_terminal()
+            )
+        else:
+            terminal = self.is_terminal()
+
+        # Update the counters
+        if not self.parallel or self._next_player_index == 0:
+            self._current_turn += 1
+
+        timestep = TimeStep(
+            observation=self.get_observation(),
+            reward=self.get_zero_rewards(),
+            terminal=terminal,
+        )  # Return all the messages
+        return timestep

agentreview/environments/paper_decision.py

@@ -0,0 +1,161 @@
+import logging
+import traceback
+from typing import List
+
+from agentreview.environments import Conversation
+from .base import TimeStep
+from ..message import Message, MessagePool
+
+
+logger = logging.getLogger(__name__)
+
+
+class PaperDecision(Conversation):
+    """
+    Area chairs make decision based on the meta reviews
+    """
+
+    type_name = "paper_decision"
+
+    def __init__(self,
+                 player_names: List[str],
+                 experiment_setting: dict,
+                 paper_ids: List[int] = None,
+                 metareviews: List[str] = None,
+                 parallel: bool = False,
+
+                 **kwargs):
+        """
+
+        Args:
+            paper_id (int): the id of the paper, such as 917
+            paper_decision (str): the decision of the paper, such as "Accept: notable-top-25%"
+
+        """
+
+        # Inherit from the parent class of `class Conversation`
+        super(Conversation, self).__init__(player_names=player_names, parallel=parallel, **kwargs)
+
+        self.paper_ids = paper_ids
+        self.metareviews = metareviews
+        self.parallel = parallel
+        self.experiment_setting = experiment_setting
+        self.ac_scoring_method = kwargs.get("ac_scoring_method")
+        # The "state" of the environment is maintained by the message pool
+        self.message_pool = MessagePool()
+
+        self.ac_decisions = None
+
+        self._current_turn = 0
+        self._next_player_index = 0
+        self.phase_index = 5  # "ACs make decision based on meta review" is the last phase (Phase 5)
+
+        self._phases = None
+
+    @property
+    def phases(self):
+
+        if self._phases is None:
+            self._phases = {
+                5: {
+                    "name": "ac_make_decisions",
+                    'speaking_order': ["AC"]
+                },
+            }
+        return self._phases
+
+    def step(self, player_name: str, action: str) -> TimeStep:
+        """
+        Step function that is called by the arena.
+
+        Args:
+            player_name: the name of the player that takes the action
+            action: the action that the agents wants to take
+        """
+
+
+
+        message = Message(
+            agent_name=player_name, content=action, turn=self._current_turn
+        )
+        self.message_pool.append_message(message)
+
+        speaking_order = self.phases[self.phase_index]["speaking_order"]
+
+        # Reached the end of the speaking order. Move to the next phase.
+
+        logging.info(f"Phase {self.phase_index}: {self.phases[self.phase_index]['name']} "
+                     f"| Player {self._next_player_index}: {speaking_order[self._next_player_index]}")
+        if self._next_player_index == len(speaking_order) - 1:
+            self._next_player_index = 0
+            logger.info(f"Phase {self.phase_index}: end of the speaking order. Move to Phase {self.phase_index + 1}.")
+            self.phase_index += 1
+            self._current_turn += 1
+        else:
+            self._next_player_index += 1
+
+        timestep = TimeStep(
+            observation=self.get_observation(),
+            reward=self.get_zero_rewards(),
+            terminal=self.is_terminal(),
+        )  # Return all the messages
+
+        return timestep
+
+
+    def check_action(self, action: str, player_name: str) -> bool:
+        """Check if the action is valid."""
+
+        if player_name.startswith("AC"):
+
+            try:
+                self.ac_decisions = self.parse_ac_decisions(action)
+
+            except:
+                traceback.print_exc()
+                return False
+
+            if not isinstance(self.ac_decisions, dict):
+                return False
+
+        return True
+
+    @property
+    def ac_decisions(self):
+        return self._ac_decisions
+
+    @ac_decisions.setter
+    def ac_decisions(self, value):
+        self._ac_decisions = value
+
+    def parse_ac_decisions(self, action: str):
+        """
+        Parse the decisions made by the ACs
+        """
+
+        lines = action.split("\n")
+
+        paper2rating = {}
+
+        paper_id, rank = None, None
+
+        for line in lines:
+
+            if line.lower().startswith("paper id:"):
+                paper_id = int(line.split(":")[1].split('(')[0].strip())
+            elif self.ac_scoring_method == "ranking" and line.lower().startswith("willingness to accept:"):
+                rank = int(line.split(":")[1].strip())
+
+            elif self.ac_scoring_method == "recommendation" and line.lower().startswith("decision"):
+                rank = line.split(":")[1].strip()
+
+
+
+            if paper_id in paper2rating:
+                raise ValueError(f"Paper {paper_id} is assigned a rank twice.")
+
+            if paper_id is not None and rank is not None:
+                paper2rating[paper_id] = rank
+                paper_id, rank = None, None
+
+        return paper2rating

agentreview/environments/paper_review.py

@@ -0,0 +1,217 @@
+import json
+import json
+import logging
+import os.path as osp
+from typing import List
+
+from agentreview.environments import Conversation
+from utility.utils import get_rebuttal_dir
+from .base import TimeStep
+from ..message import Message
+from ..paper_review_message import PaperReviewMessagePool
+
+
+logger = logging.getLogger(__name__)
+
+class PaperReview(Conversation):
+    """
+    Discussion between reviewers and area chairs.
+
+    There are several phases in the reviewing process:
+        reviewer_write_reviews: reviewers write their reviews based on the paper content.
+        author_reviewer_discussion: An author respond to comments from the reviewers.
+        reviewer_ac_discussion: reviewers and an area chair discuss the paper.
+        ac_discussion: an area chair makes the final decision.
+    """
+
+    type_name = "paper_review"
+
+    def __init__(self, player_names: List[str], paper_id: int, paper_decision: str, experiment_setting: dict, args,
+                 parallel: bool = False,
+
+                 **kwargs):
+        """
+        Args:
+            paper_id (int): the id of the paper, such as 917
+            paper_decision (str): the decision of the paper, such as "Accept: notable-top-25%"
+        """
+
+        # Inherit from the parent class of `class Conversation`
+        super(Conversation, self).__init__(player_names=player_names, parallel=parallel, **kwargs)
+        self.args = args
+        self.paper_id = paper_id
+        self.paper_decision = paper_decision
+        self.parallel = parallel
+        self.experiment_setting = experiment_setting
+        self.player_to_test = experiment_setting.get('player_to_test', None)
+        self.task = kwargs.get("task")
+        self.experiment_name = args.experiment_name
+
+        # The "state" of the environment is maintained by the message pool
+        self.message_pool = PaperReviewMessagePool(experiment_setting)
+
+        self.phase_index = 0
+        self._phases = None
+
+    @property
+    def phases(self):
+
+        if self._phases is not None:
+            return self._phases
+
+        reviewer_names = [name for name in self.player_names if name.startswith("Reviewer")]
+
+        num_reviewers = len(reviewer_names)
+
+        reviewer_names = [f"Reviewer {i}" for i in range(1, num_reviewers + 1)]
+
+        self._phases = {
+            # In phase 0, no LLM-based agents are called.
+            0: {
+                "name": "paper_extraction",
+                'speaking_order': ["Paper Extractor"],
+            },
+
+            1: {
+                "name": 'reviewer_write_reviews',
+                'speaking_order': reviewer_names
+            },
+
+            # The author responds to each reviewer's review
+            2: {
+                'name': 'author_reviewer_discussion',
+                'speaking_order': ["Author" for _ in reviewer_names],
+            },
+
+            3: {
+                'name': 'reviewer_ac_discussion',
+                'speaking_order': ["AC"] + reviewer_names,
+            },
+
+            4: {
+                'name': 'ac_write_metareviews',
+                'speaking_order': ["AC"]
+            },
+            5: {
+                'name': 'ac_makes_decisions',
+                'speaking_order': ["AC"]
+            },
+        }
+
+        return self.phases
+
+    @phases.setter
+    def phases(self, value):
+        self._phases = value
+
+    def reset(self):
+        self._current_phase = "review"
+        self.phase_index = 0
+        return super().reset()
+
+
+
+    def load_message_history_from_cache(self):
+        if self._phase_index == 0:
+
+            print("Loading message history from BASELINE experiment")
+
+            full_paper_discussion_path = get_rebuttal_dir(paper_id=self.paper_id,
+                                                          experiment_name="BASELINE",
+                                                          model_name=self.args.model_name,
+                                                          conference=self.args.conference)
+
+            messages = json.load(open(osp.join(full_paper_discussion_path, f"{self.paper_id}.json"), 'r',
+                                      encoding='utf-8'))['messages']
+
+            num_messages_from_AC = 0
+
+            for msg in messages:
+
+                # We have already extracted contents from the paper.
+                if msg['agent_name'] == "Paper Extractor":
+                    continue
+
+                # Encountering the 2nd message from the AC. Stop loading messages.
+                if msg['agent_name'] == "AC" and num_messages_from_AC == 1:
+                    break
+
+                if msg['agent_name'] == "AC":
+                    num_messages_from_AC += 1
+
+                message = Message(**msg)
+                self.message_pool.append_message(message)
+
+            num_unique_reviewers = len(
+                set([msg['agent_name'] for msg in messages if msg['agent_name'].startswith("Reviewer")]))
+
+            assert num_unique_reviewers == self.args.num_reviewers_per_paper
+
+            self._phase_index = 4
+
+    def step(self, player_name: str, action: str) -> TimeStep:
+        """
+        Step function that is called by the arena.
+
+        Args:
+            player_name: the name of the player that takes the action
+            action: the action that the agents wants to take
+        """
+
+        message = Message(
+            agent_name=player_name, content=action, turn=self._current_turn
+        )
+        self.message_pool.append_message(message)
+
+        speaking_order = self.phases[self.phase_index]["speaking_order"]
+
+        # Reached the end of the speaking order. Move to the next phase.
+        logging.info(f"Phase {self.phase_index}: {self.phases[self._phase_index]['name']} "
+                     f"| Player {self._next_player_index}: {speaking_order[self._next_player_index]}")
+
+        terminal = self.is_terminal()
+
+        if self._next_player_index == len(speaking_order) - 1:
+            self._next_player_index = 0
+
+            if self.phase_index == 4:
+                terminal = True
+                logger.info(
+                    "Finishing the simulation for Phase I - IV. Please run `python run_paper_decision_cli.py ` for "
+                           "Phase V. (AC makes decisions).")
+
+            else:
+                logger.info(f"Phase {self.phase_index}: end of the speaking order. Move to Phase ({self.phase_index + 1}).")
+                self.phase_index += 1
+                self._current_turn += 1
+
+
+
+
+        else:
+            self._next_player_index += 1
+
+        timestep = TimeStep(
+            observation=self.get_observation(),
+            reward=self.get_zero_rewards(),
+            terminal=terminal,
+        )  # Return all the messages
+
+        return timestep
+
+    def get_next_player(self) -> str:
+        """Get the next player in the current phase."""
+        speaking_order = self.phases[self.phase_index]["speaking_order"]
+        next_player = speaking_order[self._next_player_index]
+        return next_player
+
+    def get_observation(self, player_name=None) -> List[Message]:
+        """Get observation for the player."""
+        if player_name is None:
+            return self.message_pool.get_all_messages()
+        else:
+
+            return self.message_pool.get_visible_messages_for_paper_review(
+                player_name, phase_index=self.phase_index, next_player_idx=self._next_player_index,
+                player_names=self.player_names
+            )

agentreview/experiment_config.py

@@ -0,0 +1,244 @@
+"""
+BASELINE: The default settings which all other settings compare against.
+
+"""
+
+baseline_setting = {
+    "AC": [
+        "BASELINE"
+    ],
+
+    "reviewer": [
+        "BASELINE",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+benign_Rx1_setting = {
+    "AC": [
+        "BASELINE"
+    ],
+
+    "reviewer": [
+        "benign",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+malicious_Rx1_setting = {
+    "AC": [
+        "BASELINE"
+    ],
+
+    "reviewer": [
+        "malicious",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+unknowledgeable_Rx1_setting = {
+    "AC": [
+        "BASELINE"
+    ],
+
+    "reviewer": [
+        "knowledgeable",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+knowledgeable_Rx1_setting = {
+    "AC": [
+        "BASELINE"
+    ],
+
+    "reviewer": [
+        "knowledgeable",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+
+responsible_Rx1_setting = {
+    "AC": [
+        "BASELINE"
+    ],
+
+    "reviewer": [
+        "responsible",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+irresponsible_Rx1_setting = {
+    "AC": [
+        "BASELINE"
+    ],
+
+    "reviewer": [
+        "irresponsible",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+conformist_ACx1_setting = {
+    "AC": [
+        "conformist"
+    ],
+
+    "reviewer": [
+        "BASELINE",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+authoritarian_ACx1_setting = {
+    "AC": [
+        "authoritarian"
+    ],
+
+    "reviewer": [
+        "BASELINE",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+inclusive_ACx1_setting = {
+    "AC": [
+        "inclusive"
+    ],
+
+    "reviewer": [
+        "BASELINE",
+        "BASELINE",
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": ['reviewer', 'ac'],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+
+
+no_numeric_ratings_setting = {
+    "AC": [
+        "BASELINE"
+    ],
+
+    "reviewer": [
+        "BASELINE"
+    ],
+
+    "author": [
+        "BASELINE"
+    ],
+    "global_settings":{
+        "provides_numeric_rating": [],
+        "persons_aware_of_authors_identities": []
+    }
+}
+
+
+# All experimental settings.
+# Customize your own by adding new settings to this dict.
+all_settings = {
+    "BASELINE": baseline_setting,
+    "benign_Rx1": benign_Rx1_setting,
+    "malicious_Rx1": malicious_Rx1_setting,
+    "knowledgeable_Rx1_setting": knowledgeable_Rx1_setting,
+    "unknowledgeable_Rx1_setting": unknowledgeable_Rx1_setting,
+    "responsible_Rx1_setting": responsible_Rx1_setting,
+    "irresponsible_Rx1_setting": irresponsible_Rx1_setting,
+    "conformist_ACx1": conformist_ACx1_setting,
+    "authoritarian_ACx1": authoritarian_ACx1_setting,
+    "inclusive_ACx1": inclusive_ACx1_setting,
+    "no_numeric_ratings": no_numeric_ratings_setting,
+
+}
+

agentreview/message.py

@@ -0,0 +1,150 @@
+import hashlib
+import time
+from dataclasses import dataclass
+from typing import List, Union
+from uuid import uuid1
+
+# Preserved roles
+SYSTEM_NAME = "System"
+MODERATOR_NAME = "Moderator"
+
+
+def _hash(input: str):
+    """
+    Helper function that generates a SHA256 hash of a given input string.
+
+    Parameters:
+        input (str): The input string to be hashed.
+
+    Returns:
+        str: The SHA256 hash of the input string.
+    """
+    hex_dig = hashlib.sha256(input.encode()).hexdigest()
+    return hex_dig
+
+
+@dataclass
+class Message:
+    """
+    Represents a message in the chatArena environment.
+
+    Attributes:
+        agent_name (str): Name of the agent who sent the message.
+        content (str): Content of the message.
+        turn (int): The turn at which the message was sent.
+        timestamp (int): Wall time at which the message was sent. Defaults to current time in nanoseconds.
+        visible_to (Union[str, List[str]]): The receivers of the message. Can be a single agent, multiple agents, or 'all'. Defaults to 'all'.
+        msg_type (str): Type of the message, e.g., 'text'. Defaults to 'text'.
+        logged (bool): Whether the message is logged in the database. Defaults to False.
+    """
+
+    agent_name: str
+    content: str
+    turn: int
+    timestamp: int = time.time_ns()
+    visible_to: Union[str, List[str]] = "all"
+    msg_type: str = "text"
+    logged: bool = False  # Whether the message is logged in the database
+
+    @property
+    def msg_hash(self):
+        # Generate a unique message id given the content, timestamp and role
+        return _hash(
+            f"agent: {self.agent_name}\ncontent: {self.content}\ntimestamp: {str(self.timestamp)}\nturn: {self.turn}\nmsg_type: {self.msg_type}"
+        )
+
+
+class MessagePool:
+    """
+    A pool to manage the messages in the chatArena environment.
+
+    The pool is essentially a list of messages, and it allows a unified treatment of the visibility of the messages.
+    It supports two configurations for step definition: multiple players can act in the same turn (like in rock-paper-scissors).
+    Agents can only see the messages that 1) were sent before the current turn, and 2) are visible to the current role.
+    """
+
+    def __init__(self):
+        """Initialize the MessagePool with a unique conversation ID."""
+        self.conversation_id = str(uuid1())
+        self._messages: List[
+            Message
+        ] = []
+        self._last_message_idx = 0
+
+    def reset(self):
+        """Clear the message pool."""
+        self._messages = []
+
+    def append_message(self, message: Message):
+        """
+        Append a message to the pool.
+
+        Parameters:
+            message (Message): The message to be added to the pool.
+        """
+        self._messages.append(message)
+
+    def print(self):
+        """Print all the messages in the pool."""
+        for message in self._messages:
+            print(f"[{message.agent_name}->{message.visible_to}]: {message.content}")
+
+    @property
+    def last_turn(self):
+        """
+        Get the turn of the last message in the pool.
+
+        Returns:
+            int: The turn of the last message.
+        """
+        if len(self._messages) == 0:
+            return 0
+        else:
+            return self._messages[-1].turn
+
+    @property
+    def last_message(self):
+        """
+        Get the last message in the pool.
+
+        Returns:
+            Message: The last message.
+        """
+        if len(self._messages) == 0:
+            return None
+        else:
+            return self._messages[-1]
+
+    def get_all_messages(self) -> List[Message]:
+        """
+        Get all the messages in the pool.
+
+        Returns:
+            List[Message]: A list of all messages.
+        """
+        return self._messages
+
+    def get_visible_messages(self, agent_name, turn: int) -> List[Message]:
+        """
+        Get all the messages that are visible to a given agent before a specified turn.
+
+        Parameters:
+            agent_name (str): The name of the agent.
+            turn (int): The specified turn.
+
+        Returns:
+            List[Message]: A list of visible messages.
+        """
+
+        # Get the messages before the current turn
+        prev_messages = [message for message in self._messages if message.turn < turn]
+
+        visible_messages = []
+        for message in prev_messages:
+            if (
+                message.visible_to == "all"
+                or agent_name in message.visible_to
+                or agent_name == "Moderator"
+            ):
+                visible_messages.append(message)
+        return visible_messages

agentreview/paper_processor.py

@@ -0,0 +1,163 @@
+"""
+Read papers from a PDF file and extract the title, abstract, figures and tables captions, and main content. These
+functions work best with ICLR / NeurIPS papers.
+
+"""
+
+from io import StringIO
+
+from pdfminer.converter import TextConverter
+from pdfminer.layout import LAParams
+from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
+from pdfminer.pdfpage import PDFPage
+
+
+def extract_text_from_pdf(path: str) -> str:
+    """Extracts text from a PDF file.
+
+    Args:
+        path (str): A string specifying the path to the PDF file.
+
+    Returns:
+        A string containing the extracted text from the PDF.
+    """
+
+    with open(path, 'rb') as file_handle:
+        # Initialize a PDF resource manager to store shared resources.
+        resource_manager = PDFResourceManager()
+
+        # Set up a StringIO instance to capture the extracted text.
+        text_output = StringIO()
+
+        # Create a TextConverter to convert PDF pages to text.
+        converter = TextConverter(resource_manager, text_output, laparams=LAParams())
+
+        # Initialize a PDF page interpreter.
+        interpreter = PDFPageInterpreter(resource_manager, converter)
+
+        # Process each page in the PDF.
+        for page in PDFPage.get_pages(file_handle, caching=True, check_extractable=True):
+            interpreter.process_page(page)
+
+        # Retrieve the extracted text and close the StringIO instance.
+        extracted_text = text_output.getvalue()
+        text_output.close()
+
+        # Finalize the converter.
+        converter.close()
+
+    # Replace form feed characters with newlines.
+    extracted_text = extracted_text.replace('\x0c', '\n')
+
+    return extracted_text
+
+
+def convert_text_into_dict(text: str) -> dict:
+    """Converts the extracted text into a dictionary.
+
+    Args:
+        text (str): the extracted text from the PDF.
+
+    Returns:
+        A json object containing the extracted fields from the paper.
+
+    """
+
+    lines = text.split('\n')
+
+    # Create a filtered list to store non-matching lines
+    filtered_lines = [line for line in lines if not (line.startswith('Under review') or
+                                                     line.startswith('Published as') or
+                                                     line.startswith('Paper under double-blind review'))]
+
+    # Remove the first few empty lines before the title
+    while filtered_lines[0].strip() == "":
+        filtered_lines.pop(0)
+
+    # Get title
+    title = ""
+    while filtered_lines[0] != "":
+        title += filtered_lines.pop(0) + ' '
+
+    title = title.strip().capitalize()
+
+    # Remove the author information between the title and the abstract
+    while filtered_lines[0].lower() != "abstract":
+        filtered_lines.pop(0)
+    filtered_lines.pop(0)
+
+    # Get abstract
+    abstract = ""
+    while filtered_lines[0].lower() != "introduction":
+        abstract += filtered_lines.pop(0) + ' '
+
+    main_content = ""
+
+    figures_captions = []
+    tables_captions = []
+
+    while filtered_lines != [] and not filtered_lines[0].lower().startswith("references"):
+        figure_caption = ""
+        table_caption = ""
+
+        if filtered_lines[0].lower().startswith("figure"):
+            while not filtered_lines[0] == "":
+                figure_caption += filtered_lines.pop(0) + ' '
+
+
+        elif filtered_lines[0].lower().startswith("Table"):
+            while not filtered_lines[0] == "":
+                table_caption += filtered_lines.pop(0) + ' '
+
+        else:
+            main_content += filtered_lines.pop(0) + ' '
+
+        if figure_caption != "":
+            figures_captions.append(figure_caption)
+
+        if table_caption != "":
+            tables_captions.append(table_caption)
+
+
+    figures_captions = "\n".join(figures_captions) + "\n" + "\n".join(tables_captions)
+
+    # Get the first section title in the Appendix
+    # Example section title: "A ENVIRONMENT DETAILS"
+    while filtered_lines != [] and not (filtered_lines[0].isupper() and filtered_lines[0][0] == "A"):
+        filtered_lines.pop(0)
+
+
+    appendix = ""
+
+    while filtered_lines != []:
+        appendix += filtered_lines.pop(0) + ' '
+
+    # Now we have reached the "References" section
+    # Skip until we reach
+
+
+    paper = {
+        "Title": title.strip(),
+        "Abstract": abstract.strip(),
+        "Figures/Tables Captions": figures_captions.strip(),
+        "Main Content": main_content.strip(),
+        "Appendix": appendix.strip(),
+    }
+
+    return paper
+
+
+if __name__ == "__main__":
+    from utility.authentication_utils import read_and_set_openai_key
+    from agentreview.review import get_lm_review
+
+    read_and_set_openai_key()
+
+    path = "data/rejected/6359.pdf"
+    text = extract_text_from_pdf(path)
+
+    parsed_paper = convert_text_into_dict(text)
+
+    review_generated = get_lm_review(parsed_paper)
+
+    print(review_generated["review_generated"])

agentreview/paper_review_arena.py

@@ -0,0 +1,185 @@
+import csv
+import glob
+import json
+import logging
+import os
+from typing import Union
+
+from agentreview.arena import Arena, TooManyInvalidActions
+from agentreview.role_descriptions import get_reviewer_description
+from utility.utils import get_next_review_id, get_reviewer_type_from_profile, \
+    get_paper_review_and_rebuttal_dir, format_metareviews
+from .agent import Player
+from .config import ArenaConfig
+from .environments import TimeStep, load_environment
+from .paper_review_player import PaperExtractorPlayer, AreaChair, Reviewer
+
+
+logger = logging.getLogger(__name__)
+
+
+class PaperReviewArena(Arena):
+    """Arena for the paper review environment.
+
+    """
+
+    # PaperReviewArena.from_config
+    @classmethod
+    def from_config(cls, config: Union[str, ArenaConfig]):
+        """Create an arena from a config."""
+        # If config is a path, load the config
+        if isinstance(config, str):
+            config = ArenaConfig.load(config)
+
+        global_prompt = config.get("global_prompt", None)
+
+        # Create the players
+        players = []
+        for player_config in config.players:
+            # Add public_prompt to the player config
+            if global_prompt is not None:
+                player_config["global_prompt"] = global_prompt
+
+            if player_config['name'].startswith("Paper Extractor"):
+                player = PaperExtractorPlayer.from_config(player_config)
+
+            elif player_config['name'].startswith("AC"):
+                player = AreaChair.from_config(player_config)
+
+            elif player_config['name'].startswith("Reviewer"):
+                player = Reviewer.from_config(player_config)
+
+            else:
+                player = Player.from_config(player_config)
+            players.append(player)
+
+        # Check that the player names are unique
+        player_names = [player.name for player in players]
+        assert len(player_names) == len(
+            set(player_names)
+        ), f"Player names must be unique, current players: {[','.join(player_names)]}"
+
+        # Create the environment
+        config.environment[
+            "player_names"
+        ] = player_names  # add the player names to the environment config
+        env = load_environment(config.environment)
+
+        return cls(players, env, global_prompt=global_prompt)
+
+    # PaperReviewArena.step()
+    def step(self) -> TimeStep:
+        """Take a step in the game: one player takes an action and the environment updates."""
+
+        # if self.environment.phase_index > 4 and self.args.task == "paper_review":
+        #     logger.info("Finishing the simulation for Phase I - IV. Please run `python run_paper_decision_cli.py ` for "
+        #           "Phase V. (AC makes decisions).")
+        #     return
+        #
+        # elif self.environment.phase_index > 5 and self.args.task == "paper_decision":
+        #     logger.info("Finishing the simulation for Phase V. (AC makes decisions).")
+        #     return
+
+        player_name = self.environment.get_next_player()
+
+        player = self.name_to_player[player_name]  # get the player object
+
+        observation = self.environment.get_observation(
+            player_name
+        )  # get the observation for the player
+
+        timestep = None
+
+        # try to take an action for a few times
+        for i in range(self.invalid_actions_retry):
+
+
+            # Update reviewer description for rebuttal
+            if self.environment.phase_index == 3 and player.name.startswith("Reviewer"):
+                logging.info("Update reviewers' role_desc for Phase 3 (reviewer_ac_discussion)")
+                reviewer_index = int(player.name.split("Reviewer ")[1])
+
+                # reviewer_index starts from 1, so we need to subtract 1 to get the index of the reviewer in the list
+
+                player.role_desc = get_reviewer_description(phase="reviewer_ac_discussion",
+                                                            **self.environment.experiment_setting["players"][
+                                                                'Reviewer'][reviewer_index - 1])
+
+            elif self.environment.phase_index == 5:  # Phase 5 AC Makes Decisions
+
+                player.role_desc += format_metareviews(self.environment.metareviews, self.environment.paper_ids)
+
+            action = player(observation)  # take an action
+
+            if self.environment.check_action(action, player_name):  # action is valid
+                timestep = self.environment.step(
+                    player_name, action
+                )  # update the environment
+                break
+            else:  # action is invalid
+                logging.warning(f"{player_name} made an invalid action {action}")
+                continue
+
+        if (
+                timestep is None
+        ):  # if the player made invalid actions for too many times, terminate the game
+            warning_msg = f"{player_name} has made invalid actions for {self.invalid_actions_retry} times. Terminating the game."
+            logging.warning(warning_msg)
+            raise TooManyInvalidActions(warning_msg)
+
+        return timestep
+
+    def save_history(self, path: str):
+        """
+        Save the history of the game to a file.
+
+        Supports csv and json formats.
+        """
+        messages = self.environment.get_observation()
+        message_rows = []
+
+        if path.endswith(".csv"):
+            header = [
+                "agent_name",
+                "content",
+                "turn",
+                "timestamp",
+                "visible_to",
+                "msg_type",
+            ]
+            for message in messages:
+                message_row = [
+                    message.agent_name,
+                    message.content,
+                    message.turn,
+                    str(message.timestamp),
+                    message.visible_to,
+                    message.msg_type,
+                ]
+                message_rows.append(message_row)
+
+            with open(path, "w") as f:
+                writer = csv.writer(f)
+                writer.writerow(header)
+                writer.writerows(message_rows)
+        elif path.endswith(".json"):
+            for message in messages:
+                message_row = {
+                    "agent_name": message.agent_name,
+                    "content": message.content,
+                    "turn": message.turn,
+                    "timestamp": str(message.timestamp),
+                    "visible_to": message.visible_to,
+                    "msg_type": message.msg_type,
+                }
+                message_rows.append(message_row)
+
+            with open(path, "w") as f:
+
+
+                json.dump({
+                    "experiment_setting": self.environment.experiment_setting,
+                    "messages": message_rows,
+                }, f, indent=2)
+        else:
+            raise ValueError("Invalid file format")

agentreview/paper_review_message.py

@@ -0,0 +1,104 @@
+import logging
+from typing import List
+
+from agentreview.message import MessagePool, Message
+
+
+class PaperReviewMessagePool(MessagePool):
+    """
+    A pool to manage the messages in the paper review environment.
+
+    """
+
+    def __init__(self, experiment_setting: dict):
+        super().__init__()
+        self.experiment_setting = experiment_setting
+
+
+    def get_visible_messages_for_paper_review(self, agent_name, phase_index: int,
+                                              next_player_idx: int, player_names: List[str]) \
+            -> (List)[Message]:
+        """
+        Get all the messages that are visible to a given agent before a specified turn.
+
+        Parameters:
+            agent_name (str): The name of the agent.
+            turn (int): The specified turn.
+            phase_index (int): The specified phase in paper reviewing process.
+
+        Returns:
+            List[Message]: A list of visible messages.
+        """
+
+        reviewer_names = sorted([name for name in player_names if name.startswith("Reviewer")])
+
+        # Get the messages before the current turn
+        # prev_messages = [message for message in self._messages if message.turn < turn]
+        prev_messages = self._messages
+
+        if phase_index in [0, 1]:
+            visible_messages = [message for message in prev_messages if message.agent_name == "Paper Extractor"]
+
+        elif phase_index == 2:
+            visible_messages = []
+
+            for message in prev_messages:
+
+                # The author can see the paper content and each reviewer's review
+                if message.agent_name == "Paper Extractor" or message.agent_name == reviewer_names[next_player_idx]:
+                    visible_messages.append(message)
+
+            # raise NotImplementedError(f"In Phase {phase_index}, only authors can respond to reviewers' "
+            #                           f"reviews, but the current agent is {agent_name}.")
+
+        elif phase_index == 3:
+            if [agent_name.startswith(prefix) for prefix in ["AC", "Reviewer", "Paper Extractor"]]:
+                # Both area chairs and reviewers can see all the reviews and rebuttals
+                visible_messages = prev_messages
+
+            elif agent_name.startswith("Author"):
+                visible_messages = []
+
+        elif phase_index == 4:
+            if agent_name.startswith("AC"):
+                area_chair_type = self.experiment_setting['players']['AC'][0]["area_chair_type"]
+
+                # 'BASELINE' means we do not specify the area chair's characteristics in the config file
+                if area_chair_type in ["inclusive", "BASELINE"]:
+                    # An inclusive area chair can see all the reviews and rebuttals
+                    visible_messages = prev_messages
+
+                elif area_chair_type == "conformist":
+                    visible_messages = []
+
+                    for message in prev_messages:
+                        if message.agent_name.startswith("Author") or message.agent_name.startswith("Reviewer"):
+                            visible_messages.append(message)
+
+
+                elif area_chair_type == "authoritarian":
+                    visible_messages = []
+
+                    for message in prev_messages:
+                        if not (message.agent_name.startswith("Author") or message.agent_name.startswith("Reviewer")):
+                            visible_messages.append(message)
+
+                else:
+                    raise ValueError(f"Unknown Area chair type: {area_chair_type}.")
+
+
+        else:
+
+            visible_messages = []
+            for message in prev_messages:
+                if (
+                        message.visible_to == "all"
+                        or agent_name in message.visible_to
+                        or agent_name == "Moderator"
+                ):
+                    visible_messages.append(message)
+
+        logging.info(f"Phase {phase_index}： {agent_name} sees {len(visible_messages)} messages from "
+                     f"{','.join([agent.agent_name for agent in visible_messages]) if visible_messages else 'None'}")
+
+        return visible_messages

agentreview/paper_review_player.py

@@ -0,0 +1,120 @@
+import logging
+import logging
+import os
+from pathlib import Path
+from typing import List, Union
+
+from llama_index.readers.file.docs import PDFReader
+
+from agentreview.agent import Player
+from .backends import IntelligenceBackend
+from .config import BackendConfig
+from .message import Message
+
+
+class AreaChair(Player):
+
+    def __init__(
+            self,
+            name: str,
+            role_desc: str,
+            env_type: str,
+            backend: Union[BackendConfig, IntelligenceBackend],
+            global_prompt: str = None,
+            **kwargs,
+    ):
+        super().__init__(name, role_desc, backend, global_prompt, **kwargs)
+        self.env_type = env_type
+        self.role_desc = role_desc
+
+    def act(self, observation: List[Message]) -> str:
+
+        # The author just finished their rebuttals (so last speaker is Author 1).
+        # The AC asks each reviewer to update their reviews.
+
+        if self.env_type == "paper_review":
+            if len(observation) > 0 and observation[-1].agent_name.startswith("Author"):
+                return "Dear reviewers, please update your reviews based on the author's rebuttals."
+
+            else:
+                return super().act(observation)
+
+        elif self.env_type == "paper_decision":
+            return super().act(observation)
+
+        else:
+            raise ValueError(f"Unknown env_type: {self.env_type}")
+
+
+class Reviewer(Player):
+
+    def __init__(
+            self,
+            name: str,
+            role_desc: str,
+            backend: Union[BackendConfig, IntelligenceBackend],
+            global_prompt: str = None,
+            **kwargs,
+    ):
+        super().__init__(name, role_desc, backend, global_prompt, **kwargs)
+
+    def act(self, observation: List[Message]) -> str:
+        return super().act(observation)
+
+
+class PaperExtractorPlayer(Player):
+    """A player for solely extracting contents from a paper.
+
+    No API calls are made by this player.
+    """
+
+    def __init__(
+            self,
+            name: str,
+            role_desc: str,
+            paper_id: int,
+            paper_decision: str,
+            conference: str,
+            backend: Union[BackendConfig, IntelligenceBackend],
+            global_prompt: str = None,
+            **kwargs,
+    ):
+        super().__init__(name, role_desc, backend, global_prompt, **kwargs)
+        self.paper_id = paper_id
+        self.paper_decision = paper_decision
+        self.conference: str = conference
+
+    def act(self, observation: List[Message]) -> str:
+        """
+        Take an action based on the observation (Generate a response), which can later be parsed to actual actions that affect the game dynamics.
+
+        Parameters:
+            observation (List[Message]): The messages that the player has observed from the environment.
+
+        Returns:
+            str: The action (response) of the player.
+        """
+        print("Improve paper loading")
+        logging.info(f"Loading {self.conference} paper {self.paper_id} ({self.paper_decision}) ...")
+
+        loader = PDFReader()
+        document_path = Path(os.path.join(self.args.data_dir, self.conference, "paper", self.paper_decision,
+                                          f"{self.paper_id}.pdf"))  #
+        documents = loader.load_data(file=document_path)
+
+        num_words = 0
+        main_contents = "Contents of this paper:\n\n"
+        FLAG = False
+
+        for doc in documents:
+            text = doc.text.split(' ')
+            if len(text) + num_words > self.args.max_num_words:
+                text = text[:self.args.max_num_words - num_words]
+                FLAG = True
+            num_words += len(text)
+            text = " ".join(text)
+            main_contents += text + ' '
+            if FLAG:
+                break
+
+        return main_contents

agentreview/paper_review_settings.py

@@ -0,0 +1,114 @@
+default_reviewer_setting = {
+    "is_benign": None,
+    "is_knowledgeable": None,
+    "is_responsible": None,
+    "provides_numeric_rating": True,
+}
+
+
+def get_experiment_settings(setting: dict):
+    """
+    Generate experiment settings based on provided configurations for area chairs (AC) and reviewers.
+
+    Args:
+        setting (dict): A dictionary containing configuration for AC, reviewers, authors, and global settings.
+
+    Returns:
+        dict: Experiment settings including players (Paper Extractor, AC, Author, Reviewer)
+              and global settings.
+    """
+
+    experiment_setting = {
+        "id": None,
+        "players": {
+
+            # Paper Extractor is a special player that extracts a paper from the dataset.
+            # Its constructor does not take any arguments.
+            "Paper Extractor": [{}],
+
+            # Assume there is only one area chair (AC) in the experiment.
+            "AC": [get_ac_setting_from_ac_type(ac_type) for ac_type in setting['AC']],
+
+            # Author role with default configuration.
+            "Author": [{}],
+
+            # Reviewer settings are generated based on reviewer types provided in the settings.
+            "Reviewer": [get_reviewer_setting_from_reviewer_type(reviewer_type) for reviewer_type in setting[
+                'reviewer']],
+        },
+        "global_settings": setting['global_settings']
+    }
+
+    return experiment_setting
+
+
+def get_reviewer_setting_from_reviewer_type(reviewer_type: str):
+    """
+    Map a reviewer type (e.g., 'benign', 'malicious') to a reviewer setting dictionary.
+
+    Args:
+        reviewer_type (str): The type of reviewer (e.g., 'benign', 'malicious', 'knowledgeable').
+
+    Returns:
+        dict: A dictionary representing the reviewer's attributes like is_benign, is_knowledgeable,
+              is_responsible, or if they know the authors (e.g., 'famous', 'unfamous').
+
+    Raises:
+        ValueError: If an unknown reviewer type is provided.
+    """
+    reviewer_setting = {
+        "is_benign": None,
+        "is_knowledgeable": None,
+        "is_responsible": None
+    }
+
+    # Intention
+    if reviewer_type == "benign":
+        reviewer_setting["is_benign"] = True
+    elif reviewer_type == "malicious":
+        reviewer_setting["is_benign"] = False
+
+    # Knowledgeability
+    elif reviewer_type == "knowledgeable":
+        reviewer_setting["is_knowledgeable"] = True
+    elif reviewer_type == "unknowledgeable":
+        reviewer_setting["is_knowledgeable"] = False
+
+    # Commitment
+    elif reviewer_type == "responsible":
+        reviewer_setting["is_responsible"] = True
+    elif reviewer_type == "irresponsible":
+        reviewer_setting["is_responsible"] = False
+
+    elif reviewer_type in ["BASELINE"]:
+        pass
+
+    elif reviewer_type in ["authors_are_famous"]:
+        reviewer_setting["knows_authors"] = "famous"
+
+    elif reviewer_type in ["authors_are_unfamous"]:
+        reviewer_setting["knows_authors"] = "unfamous"
+
+
+    else:
+        raise ValueError(f"Unknown reviewer type: {reviewer_type}")
+
+    return reviewer_setting
+
+
+def get_ac_setting_from_ac_type(ac_type: str):
+    """
+    Generate the area chair (AC) settings based on the type of AC.
+
+    Args:
+        ac_type (str): The type of area chair (e.g., 'senior', 'junior').
+
+    Returns:
+        dict: A dictionary containing the area chair type.
+    """
+
+    ac_setting = {
+        "area_chair_type": ac_type
+    }
+
+    return ac_setting

agentreview/role_descriptions.py

@@ -0,0 +1,515 @@
+import os
+import sys
+
+import numpy as np
+
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
+
+import const
+from agentreview.config import AgentConfig
+
+PLAYER_BACKEND = {
+    "backend_type": "openai-chat",
+    "temperature": 0.9,
+    "max_tokens": 4096
+}
+
+# archived. If we use this rubric, the scores given by the reviewers are too high
+RUBRICS_v1 = ("Rubrics: 10 for strong accept (top 5% of accepted papers), "
+              "8 for accept (top 50% of accepted papers), "
+              "6 for borderline accept, "
+              "5 for borderline reject, "
+              "3 for reject, and 1 for strong reject. ")
+
+SCORE_CALCULATION_v1 = {
+    10: "This study is among the top 0.5% of all papers",
+    8: "This study is one of the most thorough I have seen. It changed my thinking on this topic. I would fight for it to be accepted",
+    6: "This study provides sufficient support for all of its claims/arguments. Some extra experiments are needed, but not essential. The method is highly original and generalizable to various fields. It deepens the understanding of some phenomenons or lowers the barriers to an existing research direction",
+    5: "This study provides sufficient support for its major claims/arguments, some minor points may need extra support or details. The method is moderately original and generalizable to various relevant fields. The work it describes is not particularly interesting and/or novel, so it will not be a big loss if people don’t see it in this conference",
+    3: "Some of the main claims/arguments are not sufficiently supported, there are major technical/methodological problems. The proposed method is somewhat original and generalizable to various relevant fields. I am leaning towards rejection, but I can be persuaded if my co-reviewers think otherwise",
+    1: "This study is not yet sufficiently thorough to warrant publication or is not relevant to the conference. This paper makes marginal contributions"
+}
+
+# Start to use this rubric from 2024.1.23 as SCORE_CALCULATION_v1 is too harsh
+SCORE_CALCULATION = {
+    10: "This study is among the top 2% of all papers. It is one of the most thorough I have seen. It changed my "
+        "thinking on this topic. I would fight for it to be accepted",
+    8: "This study is among the top 10% of all papers. It provides sufficient support for all of its claims/arguments. "
+       "Some extra experiments are needed, "
+       "but not essential. The method is highly original and generalizable to various fields. It deepens the understanding of some phenomenons or lowers the barriers to an existing research direction",
+    6: "This study provides sufficient support for its major claims/arguments, some minor points may need extra support or details. The method is moderately original and generalizable to various relevant fields. The work it describes is not particularly interesting and/or novel, so it will not be a big loss if people don’t see it in this conference",
+    5: "Some of the main claims/arguments are not sufficiently supported, there are major technical/methodological "
+       "problems. The proposed method is somewhat original and generalizable to various relevant fields. I am leaning towards rejection, but I can be persuaded if my co-reviewers think otherwise",
+    3: "This paper makes marginal contributions",
+    1: "This study is not yet sufficiently thorough to warrant publication or is not relevant to the conference"
+}
+
+RUBRICS_v1 = ("Rubrics: "
+              f"10 for strong accept ({SCORE_CALCULATION[10]}); "
+              f"8 for accept ({SCORE_CALCULATION[8]}); "
+              f"6 for borderline accept ({SCORE_CALCULATION[6]}); "
+              f"5 for borderline reject ({SCORE_CALCULATION[5]}); "
+              f"3 for reject ({SCORE_CALCULATION[3]}); "
+              f"1 for strong reject ({SCORE_CALCULATION[1]}); ")
+
+INSTRUCTIONS_FOR_FAMOUS_AUTHORS = ("You know that the authors of the paper are from a very famous lab and have "
+                                   "several publication in "
+                                   "this "
+                                   "field. Be sure to consider that when writing the paper reviews. "
+                                   "\n\n")
+
+RUBRICS = (f"* 10: {SCORE_CALCULATION[10]};\n"
+           f"* 8: {SCORE_CALCULATION[8]};\n"
+           f"* 6: {SCORE_CALCULATION[6]};\n"
+           f"* 5: {SCORE_CALCULATION[5]};\n"
+           f"* 3: {SCORE_CALCULATION[3]};\n"
+           f"* 1: {SCORE_CALCULATION[1]}. ")
+
+# Try to lower the score
+SCORE_CONTROL = ("This is a very rigorous top-tier conference. "
+                 "Most papers get scores <=5 before the rebuttal. ")
+
+# Need to explain this
+EXPLANATION_FOR_NOT_UPDATING_MANUSCRIPT = (f"Note: Do not mention that the authors did not update the manuscripts "
+                                           f"and do not penalize them for "
+                                           f"not revising their papers. They cannot do it now. Just assume they have revised their "
+                                           f"manuscripts according to their rebuttals.")
+
+
+def get_instructions_for_overall_scores(author_type: str) -> str:
+    instruction = "Do not write any reasons. "
+
+    if author_type not in ["famous"]:
+        instruction += ("Do not assign scores of 7 or higher before the rebuttal unless the paper "
+                        "demonstrates exceptional originality and "
+                        "significantly advances the state-of-the-art in machine learning. "
+                        )
+    instruction += "Intermediary integer scores such as 9, 7, 4, and 2 are allowed. "
+
+    return instruction
+
+def get_reviewer_description(is_benign: bool = None, is_knowledgeable: bool = None, is_responsible: bool = None,
+                             provides_numeric_rating:
+                             bool = True, knows_authors: bool = False, phase: str = "reviewer_write_reviews"):
+    assert phase in ["reviewer_write_reviews", 'reviewer_ac_discussion']
+    assert provides_numeric_rating in [True, False]
+    bio = ("You are a reviewer. You write peer review of academic papers by evaluating their technical "
+           f"quality, originality, and clarity. ")
+
+    # The reviewer's famous identities are known to the AC
+    if knows_authors:
+        bio += "\n\n" + INSTRUCTIONS_FOR_FAMOUS_AUTHORS
+
+    else:
+        bio += f"{SCORE_CONTROL}\n\n"
+
+    bio += "## Review Guidelines\n"
+
+    if phase in ["reviewer_write_reviews"]:
+
+        guideline = "Write a peer review using the following format:\n\n"
+        guideline += "```\n"
+        if provides_numeric_rating:
+            guideline += f"Overall rating: ... # {get_instructions_for_overall_scores(knows_authors)}\n\n"
+
+        """
+        # Review formats used in most ICLR conferences
+        guideline += "Summary: ... # Provide a brief summary of the paper, such as its main contributions.\n\n"
+        guideline += "Strengths: ... # Give a list of strengths for the paper.\n\n"
+        guideline += "Weaknesses: ...<EOS> # Give a list of weaknesses and questions for the paper.\n\n"
+        """
+
+        # Review formats used in [Stanford's Nature Submission](https://arxiv.org/abs/2310.01783)
+        guideline += "Significance and novelty: ... \n\n"
+        guideline += "Reasons for acceptance: ... # List 4 key reasons. \n\n"
+        guideline += "Reasons for rejection: ... # List 4 key reasons. For each of 4 key reasons, use **>=2 sub bullet points** to further clarify and support your arguments in painstaking details \n\n"
+        guideline += "Suggestions for improvement: ... <EOS> # List 4 key suggestions \n\n"
+
+
+
+
+
+
+
+    elif phase in ["reviewer_ac_discussion"]:
+
+        guideline = "Based on the authors' responses, write an updated paper review in the reviewer-AC discussion."
+
+        if provides_numeric_rating:
+            guideline += (
+                "Decrease your score if the authors fail to address your or other reviewers' concerns, or "
+                f"provide very vague responses. "
+                f"Increase your score only if the authors have "
+                f"addressed all your and other "
+                f"reviewers' concerns, and have comprehensively described how they plan to update the manuscript. Keep "
+                f"the "
+                f"score "
+                f"unchanged "
+                f"otherwise. {EXPLANATION_FOR_NOT_UPDATING_MANUSCRIPT}"
+                "\n\n## Format for the updated review\n\n```\n")
+
+            guideline += ("Overall rating: ... # Provide an updated overall rating using an integer from 1 to 10. Do "
+                          "not penalize the authors for not updating their manuscripts. They cannot revise their "
+                          "manuscripts now.")
+        else:
+            guideline += "\n\n```\n"
+
+        guideline += (f"Summary: ... <EOS> # "
+                      f"{'Provide a justification on your updated score.' if provides_numeric_rating else ''} Comment on "
+                      f"whether the "
+                      "author has "
+                      "addressed "
+                      "your questions and concerns. Note that authors cannot revise their "
+                      "manuscripts now.\n")
+
+    else:
+        raise ValueError(f"Invalid phase for a reviewer: {phase}")
+
+    bio += f"{guideline}```\n\n"
+
+    if not all([x is None for x in [is_benign, is_knowledgeable, is_responsible]]):
+        bio += "## Your Biography\n"
+
+    # Knowledgeability
+    desc_knowledgeable_reviewer = (
+        "You are knowledgeable, with a strong background and a PhD degree in the subject areas "
+        "related to this paper. "
+        "You possess the expertise necessary to scrutinize "
+        "and provide insightful feedback to this paper.")
+
+    desc_unknowledgeable_reviewer = (
+        "You are not knowledgeable and do not have strong background in the subject areas related to "
+        "this paper.")
+
+    if is_knowledgeable is not None:
+        if is_knowledgeable:
+            desc = desc_knowledgeable_reviewer
+        else:
+            desc = desc_unknowledgeable_reviewer
+
+        bio += f"Knowledgeability: {desc}\n\n"
+
+    # Responsible vs. lazy
+
+    desc_responsible_reviewer = ("As a responsible reviewer, you highly responsibly write paper reviews and actively "
+                                 "participate in reviewer-AC discussions. "
+                                 "You meticulously assess a research "
+                                 "paper's "
+                                 "technical accuracy, innovation, and relevance. You thoroughly read the paper, "
+                                 "critically analyze the methodologies, and carefully consider the paper's "
+                                 "contribution to the field. ")
+
+    desc_lazy_reviewer = ("As a lazy reviewer, your reviews tend to be superficial and hastily done. You do not like "
+                          "to discuss in the reviewer-AC discussion. "
+                          "Your assessments might overlook critical details, lack depth in analysis, "
+                          "fail to recognize novel contributions, "
+                          "or offer generic feedback that does little to advance the paper's quality.")
+
+    if is_responsible is not None:
+
+        if is_responsible:
+            desc = desc_responsible_reviewer
+        else:
+            desc = desc_lazy_reviewer
+
+        bio += f"Responsibility: {desc}\n\n"
+
+    # Benign (Good) vs. Malicious
+    desc_benign_reviewer = ("As a benign reviewer, your approach to reviewing is guided by a genuine intention "
+                            "to aid authors in enhancing their work. You provide detailed, constructive feedback, "
+                            "aimed at both validating robust research and guiding authors to refine and improve their work. "
+                            "You are also critical of technical flaws in the paper. ")
+
+    desc_malicious_reviewer = ("As a mean reviewer, your reviewing style is often harsh and overly critical, "
+                               "with a tendency towards negative bias. Your reviews may focus excessively on "
+                               "faults, sometimes overlooking the paper's merits. Your feedback can be discouraging, "
+                               "offering minimal guidance for improvement, and often aims more at rejection than constructive critique. ")
+
+    if is_benign is not None:
+
+        if is_benign:
+            desc = desc_benign_reviewer
+        else:
+            desc = desc_malicious_reviewer
+
+        bio += f"Intention: {desc}\n\n"
+
+    if provides_numeric_rating:
+        bio += f"## Rubrics for Overall Rating\n\n{RUBRICS}"
+
+    return bio
+
+
+def get_author_description() -> str:
+    bio = ("You are an author. You write research papers and submit them to conferences. During the rebuttal phase, "
+           "you carefully read the reviews from the reviewers and respond to each of them.\n\n")
+
+    bio += "## Author Guidelines\n"
+
+    bio += "Write a response to the reviews using the following format:\n\n"
+    bio += "```\n"
+    bio += ("Response: ... # Provide a brief response to each review. Address each question and weakness mentioned "
+            "by the reviewer. No need to respond to the strengths they mentioned. \n\n")
+
+    return bio
+
+
+def get_ac_description(area_chair_type: str, phase: str, scoring_method: str, num_papers_per_area_chair: int,
+                       knows_authors: bool = False, **kwargs) -> (
+        str):
+    """
+    Note: We assume that the AC definitely provides a score so that the papers can be compared
+    Args:
+        phase (str): The phase of the conference. Must be either "reviewer_ac_discussion" or "ac_write_metareviews".
+        scoring_method (str): The method used by the area chair to make the final decision. Must be either of
+            "recommendation": directly make a recommendation (e.g. "Accept", "Reject") for each paper
+            "ranking": rank the papers using your willingness to accept
+
+    """
+
+    acceptance_rate = kwargs.get('acceptance_rate', 0.32)
+    bio = "You are a very knowledgeable and experienced area chair in a top-tier machine learning conference. "
+
+    if phase == "ac_write_metareviews":
+        bio += ("You evaluate the reviews provided by reviewers and write metareviews. Later, you will decide which "
+                "paper gets accepted or rejected based on your metareviews. ")
+
+    elif phase == "ac_make_decisions":
+        bio += "Based on the metareviews you wrote previously, you decide if a paper is accepted or rejected. "
+
+    # The authors' famous identities are known to the AC
+    if knows_authors:
+        bio += INSTRUCTIONS_FOR_FAMOUS_AUTHORS + SCORE_CONTROL
+
+    bio += "\n\n## Area Chair Guidelines\n"
+
+    if phase == "ac_write_metareviews":
+
+        guideline = "Write a metareview using the following format:\n\n"
+        guideline += "```\n"
+        guideline += (
+            f"Score: ... # Provide a score for the paper in the range from 1 to 10. {get_instructions_for_overall_scores(knows_authors)}Fractions such as "
+            "6.5 is allowed.\n\n")
+        guideline += ("Summary: ... <EOS> # Provide a summary of the paper based on the paper contents (if provided), "
+                      "reviewers' "
+                      "reviews and discussions (if provided), authors' rebuttal, and your own expertise. "
+                      f"{EXPLANATION_FOR_NOT_UPDATING_MANUSCRIPT}\n")
+
+        bio += guideline
+
+        bio += "```\n\n"
+
+    elif phase == "ac_make_decisions":
+        max_num_accepted_papers = int(np.floor(num_papers_per_area_chair * acceptance_rate))
+
+        # The area chair usually accept more papers than s/he should
+        # So we use a ranking approach
+
+        if scoring_method == "recommendation":
+            num_rejected_papers = int(num_papers_per_area_chair)
+            CONTROL_NUM_ACCEPTED_PAPERS = (f"You must accept around "
+                                           f"{max_num_accepted_papers} out of {num_papers_per_area_chair} papers, "
+                                           # f"so around {num_rejected_papers - max_num_accepted_papers} papers should "
+                                           # f"have a decision of 'Reject'. "
+                                           # f"You should maintain the high criteria of this conference. "
+                                           # f"'5' is borderline reject."
+                                           )
+            guideline = (f"Carefully decide if a paper is accepted or rejected using the metareview. Use the following "
+                         f"format ")
+            guideline += f"({CONTROL_NUM_ACCEPTED_PAPERS})"
+            guideline += f":\n\n"
+
+            guideline += "```\n"
+            guideline += ("Paper ID: ... # Provide the first paper ID. \n"
+                          "Decision: ... # Provide a decision for the paper. Must be one of "
+                          "'Reject' and 'Accept'.\n"
+                          # "Reasons: ... # Provide a short justification for your decision, maximum 3 sentences. \n"
+                          "Paper ID: ... # Provide the second paper ID. \n"
+                          f"... # Likewise\n")
+            guideline += "```\n\n"
+
+            bio += guideline
+
+        elif scoring_method == "ranking":
+
+            # The area chair usually accept more papers than s/he should
+            # So we use this ranking approach
+
+            guideline = (f"Rank the papers from the paper you are most willing to accept to the least willing to "
+                         f"accept. '1' indicates "
+                         f"the paper "
+                         f"you are most "
+                         f"willing to accept. "
+                         f"Use this format:\n\n")
+            guideline += "```\n"
+            guideline += "Paper ID: 1 # The paper ID you most want to accept.\n"
+            guideline += "Willingness to accept: 1 # This integer must be unique for each paper. \n"
+            guideline += "Paper ID: ... # The second paper ID you most want to accept .. \n...\n"
+            guideline += "Willingness to accept: 2 \n"
+            guideline += "...\n```\n\n"
+
+            bio += guideline
+
+        else:
+            raise NotImplementedError(f"Unknown scoring method: {scoring_method}")
+
+
+    else:
+        raise ValueError(f"Invalid phase for an area chair: {phase}")
+
+    if phase == "ac_write_metareviews":
+        bio += f"## Rubrics for Overall Rating\n\n{RUBRICS}\n\n"
+
+    desc_inclusive_ac = ("You are an inclusive area chair. You tend to hear from all reviewers' opinions and combine "
+                         "them with your own judgments to make the final decision.")
+
+    desc_conformist_ac = ("You are a conformist area chair who perfunctorily handle area chair duties. You "
+                          "mostly follow "
+                          "the reviewers' suggestions to write your metareview, score the paper, and decide whether "
+                          "to accept a paper.")
+
+    desc_authoritarian_ac = ("You are an authoritarian area chair. You tend to read the paper on your own, follow your "
+                             "own "
+                             "judgment and mostly ignore "
+                             "the reviewers' opinions.")
+
+    desc = ""
+
+    if phase == "ac_write_metareviews":
+
+        if area_chair_type == "inclusive":
+            desc = desc_inclusive_ac
+        elif area_chair_type == "conformist":
+            desc = desc_conformist_ac
+        elif area_chair_type == "authoritarian":
+            desc = desc_authoritarian_ac
+        elif area_chair_type == "BASELINE":
+            desc = ""
+
+    elif phase == "ac_make_decisions":
+        # We do not introduce different types of ACs in the decision phase
+        desc = ""
+
+    else:
+        raise ValueError(f"Invalid area chair type: {area_chair_type}. Choose from {','.join(const.AREA_CHAIR_TYPES)}.")
+
+    if desc != "":
+        bio += f"## Your Biography\n{desc}\n\n"
+
+    return bio
+
+
+def get_reviewer_player_config(reviewer_index: int, is_benign: bool, is_knowledgeable: bool, is_responsible: bool,
+                               global_settings: dict) -> dict:
+    """
+
+    Get a Player object that represents a reviewer.
+
+    Args:
+        reviewer_index:
+        is_benign (bool): If the reviewer has good intention and provides constructive feedback. If None, we do not add this field to the bio.
+        is_knowledgeable (bool): If the reviewer is knowledgeable and has a strong background in the subject areas related
+        to the paper. If None, we do not add this field to the bio.
+        is_responsible (bool): If the reviewer is responsible and provides detailed feedback.
+        provides_numeric_rating (bool): If the reviewer provides an overall rating (e.g. accept, weak accept) to the
+        paper. If None, we do not add this field to the bio.
+        knows_authors (str): The type of the authors of the paper under review. Must be one of "famous",
+        "unfamous", None (Default. Author type is unknown)
+
+    Return
+        player (dict): A player object that represents the reviewer.
+
+    """
+
+    knows_authors = "reviewer" in global_settings['persons_aware_of_authors_identities']
+    provides_numeric_rating = "reviewer" in global_settings['provides_numeric_rating']
+
+    reviewer = {
+        "name": f"Reviewer {reviewer_index}",
+        "role_desc": get_reviewer_description(is_benign, is_knowledgeable, is_responsible, provides_numeric_rating,
+                                              knows_authors),
+        # "role_desc": get_reviewer_description(is_benign, is_knowledgeable, is_responsible, provides_numeric_rating),
+        "backend": PLAYER_BACKEND,
+        "metadata": {
+            "is_benign": is_benign,
+            "is_knowledgeable": is_knowledgeable,
+            "is_responsible": is_responsible,
+            "knows_authors": knows_authors,
+        }
+    }
+
+    return AgentConfig(**reviewer)
+
+
+def get_author_config() -> dict:
+    author = {
+        "name": f"Author",
+        "role_desc": get_author_description(),
+        "backend": PLAYER_BACKEND
+    }
+
+    return AgentConfig(**author)
+
+
+def get_paper_extractor_config(**kwargs) -> dict:
+    max_tokens = kwargs.pop('max_tokens', 2048)
+
+    paper_extractor = {
+        "name": f"Paper Extractor",
+        "role_desc": "This is a player that only extracts content from the paper. No API calls are made",
+        "backend": {
+            "backend_type": "dummy",
+            # "temperature": 0.,
+            "max_tokens": max_tokens,
+        },
+    }
+
+    return AgentConfig(**paper_extractor)
+
+
+def get_ac_config(**kwargs) -> dict:
+    """
+
+    Get a Player object that represents an area chair.
+
+    Args:
+        index_ac (int):
+        is_benign (bool): If the reviewer has good intention and provides constructive feedback.
+        is_knowledgeable: If the reviewer is knowledgeable and has a strong background in the subject areas related
+        to the paper.
+        is_responsible (bool): If the reviewer is responsible and provides detailed feedback.
+        provides_numeric_rating (bool): If the reviewer provides an overall rating (e.g. accept, weak accept) to the
+        paper.
+
+        scoring_method (str): Scoring method for the area chair.
+
+    Return
+        player (dict): A player object that represents the area chair.
+
+    """
+
+    env_type = kwargs.pop('env_type')
+    global_settings = kwargs.get('global_settings', {})
+
+    if env_type == "paper_review":
+        phase = "ac_write_metareviews"
+
+    elif env_type == "paper_decision":
+        phase = "ac_make_decisions"
+
+    else:
+        raise NotImplementedError
+
+    kwargs['phase'] = phase
+    kwargs['knows_authors'] = "ac" in global_settings['persons_aware_of_authors_identities']
+
+    area_chair = {
+        "name": "AC",  # We assume there is only 1 AC for now
+        "role_desc": get_ac_description(**kwargs),
+        "backend": {'backend_type': 'openai-chat',
+                    'temperature': 0.0,  # make the AC decision deterministic
+                    'max_tokens': 4096},
+        "env_type": env_type,
+    }
+
+    return AgentConfig(**area_chair)

agentreview/ui/cli.py

@@ -0,0 +1,269 @@
+import logging
+import logging
+import os
+import os.path as osp
+from typing import Union
+
+from colorama import Fore
+from colorama import Style as CRStyle
+from prompt_toolkit import prompt
+from prompt_toolkit.completion import WordCompleter
+from prompt_toolkit.styles import Style
+from rich.console import Console
+
+from utility.utils import get_rebuttal_dir, load_gpt4_generated_ac_decisions, \
+    save_gpt4_generated_ac_decisions
+from ..arena import Arena, TooManyInvalidActions
+from ..backends.human import HumanBackendError
+from ..environments import PaperReview, PaperDecision
+
+# Get the ASCII art from https://patorjk.com/software/taag/#p=display&f=Big&t=Chat%20Arena
+ASCII_ART = r"""
+                           _   _____            _               
+     /\                   | | |  __ \          (_)              
+    /  \   __ _  ___ _ __ | |_| |__) |_____   ___  _____      __
+   / /\ \ / _` |/ _ \ '_ \| __|  _  // _ \ \ / / |/ _ \ \ /\ / /
+  / ____ \ (_| |  __/ | | | |_| | \ \  __/\ V /| |  __/\ V  V / 
+ /_/    \_\__, |\___|_| |_|\__|_|  \_\___| \_/ |_|\___| \_/\_/  
+           __/ |                                                
+          |___/                                                                                   
+"""
+
+color_dict = {
+    "red": Fore.RED,
+    "green": Fore.GREEN,
+
+    "blue": Fore.BLUE,  # Paper Extractor
+    "light_red": Fore.LIGHTRED_EX,  # AC
+    "light_green": Fore.LIGHTGREEN_EX,  # Author
+    "yellow": Fore.YELLOW,  # R1
+    "magenta": Fore.MAGENTA,  # R2
+    "cyan": Fore.CYAN,
+    "white": Fore.WHITE,
+    "black": Fore.BLACK,
+
+    "light_yellow": Fore.LIGHTYELLOW_EX,
+    "light_blue": Fore.LIGHTBLUE_EX,
+    "light_magenta": Fore.LIGHTMAGENTA_EX,
+    "light_cyan": Fore.LIGHTCYAN_EX,
+    "light_white": Fore.LIGHTWHITE_EX,
+    "light_black": Fore.LIGHTBLACK_EX,
+
+}
+
+visible_colors = [
+    color
+    for color in color_dict  # ANSI_COLOR_NAMES.keys()
+    if color not in ["black", "white", "red", "green"] and "grey" not in color
+]
+
+try:
+    import colorama
+except ImportError:
+    raise ImportError(
+        "Please install colorama: `pip install colorama`"
+    )
+
+MAX_STEPS = 20  # We should not need this parameter for paper reviews anyway
+
+# Set logging level to ERROR
+logging.getLogger().setLevel(logging.ERROR)
+
+
+class ArenaCLI:
+    """The CLI user interface for ChatArena."""
+
+    def __init__(self, arena: Arena):
+        self.arena = arena
+        self.args = arena.args
+
+
+    def launch(self, max_steps: int = None, interactive: bool = True):
+        """Run the CLI."""
+
+        if not interactive and max_steps is None:
+            max_steps = MAX_STEPS
+
+        args = self.args
+
+        console = Console()
+        # Print ascii art
+        console.print(ASCII_ART, style="bold dark_orange3")
+        timestep = self.arena.reset()
+        console.print("🎓AgentReview Initialized!", style="bold green")
+
+        env: Union[PaperReview, PaperDecision] = self.arena.environment
+        players = self.arena.players
+
+        env_desc = self.arena.global_prompt
+        num_players = env.num_players
+        player_colors = visible_colors[:num_players]  # sample different colors for players
+        name_to_color = dict(zip(env.player_names, player_colors))
+
+        print("name_to_color: ", name_to_color)
+        # System and Moderator messages are printed in red
+        name_to_color["System"] = "red"
+        name_to_color["Moderator"] = "red"
+
+        console.print(
+            f"[bold green underline]Environment ({env.type_name}) description:[/]\n{env_desc}"
+        )
+
+        # Print the player name, role_desc and backend_type
+        for i, player in enumerate(players):
+            player_name_str = f"[{player.name} ({player.backend.type_name})] Role Description:"
+            # player_name = Text(player_name_str)
+            # player_name.stylize(f"bold {name_to_color[player.name]} underline")
+            # console.print(player_name)
+            # console.print(player.role_desc)
+
+            logging.info(color_dict[name_to_color[player.name]] + player_name_str + CRStyle.RESET_ALL)
+            logging.info(color_dict[name_to_color[player.name]] + player.role_desc + CRStyle.RESET_ALL)
+
+        console.print(Fore.GREEN + "\n========= Arena Start! ==========\n" + CRStyle.RESET_ALL)
+
+        step = 0
+        while not timestep.terminal:
+            if env.type_name == "paper_review":
+                if env.phase_index > 4:
+                    break
+
+            elif env.type_name == "paper_decision":
+                # Phase 5: AC makes decisions
+                if env.phase_index > 5:
+                    break
+
+            else:
+                raise NotImplementedError(f"Unknown environment type: {env.type_name}")
+
+            if interactive:
+                command = prompt(
+                    [("class:command", "command (n/r/q/s/h) > ")],
+                    style=Style.from_dict({"command": "blue"}),
+                    completer=WordCompleter(
+                        [
+                            "next",
+                            "n",
+                            "reset",
+                            "r",
+                            "exit",
+                            "quit",
+                            "q",
+                            "help",
+                            "h",
+                            "save",
+                            "s",
+                        ]
+                    ),
+                )
+                command = command.strip()
+
+                if command == "help" or command == "h":
+                    console.print("Available commands:")
+                    console.print("    [bold]next or n or <Enter>[/]: next step")
+                    console.print("    [bold]exit or quit or q[/]: exit the game")
+                    console.print("    [bold]help or h[/]: print this message")
+                    console.print("    [bold]reset or r[/]: reset the game")
+                    console.print("    [bold]save or s[/]: save the history to file")
+                    continue
+                elif command == "exit" or command == "quit" or command == "q":
+                    break
+                elif command == "reset" or command == "r":
+                    timestep = self.arena.reset()
+                    console.print(
+                        "\n========= Arena Reset! ==========\n", style="bold green"
+                    )
+                    continue
+                elif command == "next" or command == "n" or command == "":
+                    pass
+                elif command == "save" or command == "s":
+                    # Prompt to get the file path
+                    file_path = prompt(
+                        [("class:command", "save file path > ")],
+                        style=Style.from_dict({"command": "blue"}),
+                    )
+                    file_path = file_path.strip()
+                    # Save the history to file
+                    self.arena.save_history(file_path)
+                    # Print the save success message
+                    console.print(f"History saved to {file_path}", style="bold green")
+                else:
+                    console.print(f"Invalid command: {command}", style="bold red")
+                    continue
+
+            try:
+                timestep = self.arena.step()
+            except HumanBackendError as e:
+                # Handle human input and recover with the game update
+                human_player_name = env.get_next_player()
+                if interactive:
+                    human_input = prompt(
+                        [
+                            (
+                                "class:user_prompt",
+                                f"Type your input for {human_player_name}: ",
+                            )
+                        ],
+                        style=Style.from_dict({"user_prompt": "ansicyan underline"}),
+                    )
+                    # If not, the conversation does not stop
+                    timestep = env.step(human_player_name, human_input)
+                else:
+                    raise e  # cannot recover from this error in non-interactive mode
+            except TooManyInvalidActions as e:
+                # Print the error message
+                # console.print(f"Too many invalid actions: {e}", style="bold red")
+                print(Fore.RED + "This will be red text" + CRStyle.RESET_ALL)
+                break
+
+            # The messages that are not yet logged
+            messages = [msg for msg in env.get_observation() if not msg.logged]
+
+            # Print the new messages
+            for msg in messages:
+                message_str = f"[{msg.agent_name}->{msg.visible_to}]: {msg.content}"
+                console.print(color_dict[name_to_color[msg.agent_name]] + message_str + CRStyle.RESET_ALL)
+                msg.logged = True
+
+            step += 1
+            if max_steps is not None and step >= max_steps:
+                break
+
+        console.print("\n========= Arena Ended! ==========\n", style="bold red")
+
+        if env.type_name == "paper_review":
+
+            paper_id = self.arena.environment.paper_id
+            rebuttal_dir = get_rebuttal_dir(output_dir=self.args.output_dir,
+                                            paper_id=paper_id,
+                                            experiment_name=self.args.experiment_name,
+                                            model_name=self.args.model_name,
+                                            conference=self.args.conference)
+
+            os.makedirs(rebuttal_dir, exist_ok=True)
+
+            path_review_history = f"{rebuttal_dir}/{paper_id}.json"
+
+            if osp.exists(path_review_history):
+                raise Exception(f"History already exists!! ({path_review_history}). There must be something wrong with "
+                                f"the path to save the history ")
+
+            self.arena.save_history(path_review_history)
+
+        elif env.type_name == "paper_decision":
+            ac_decisions = load_gpt4_generated_ac_decisions(output_dir=args.output_dir,
+                                                            conference=args.conference,
+                                                            model_name=args.model_name,
+                                                            ac_scoring_method=args.ac_scoring_method,
+                                                            experiment_name=args.experiment_name,
+                                                            num_papers_per_area_chair=args.num_papers_per_area_chair)
+
+
+            ac_decisions += [env.ac_decisions]
+
+            save_gpt4_generated_ac_decisions(ac_decisions,
+                                             output_dir=args.output_dir,
+                                             conference=args.conference,
+                                             model_name=args.model_name,
+                                             ac_scoring_method=args.ac_scoring_method,
+                                             experiment_name=args.experiment_name)

agentreview/utils.py

@@ -0,0 +1,116 @@
+import json
+import re
+
+
+def is_json(myjson):
+    """
+    Checks whether a given string is a valid JSON.
+
+    Parameters:
+        myjson (str): The string to be checked.
+
+    Returns:
+        bool: True if the string is a valid JSON, False otherwise.
+    """
+    try:
+        _ = json.loads(myjson)
+    except ValueError:
+        return False
+    return True
+
+
+def is_json_inside(text):
+    """
+    Checks whether a given string contains valid JSON(s).
+
+    Parameters:
+        text (str): The string to be checked.
+
+    Returns:
+        bool: True if the string contains valid JSON(s), False otherwise.
+    """
+    text = re.sub(r"\s+", " ", text)
+    matches = re.findall(r"\{.*?\}", text)
+    for match in matches:
+        if is_json(match):
+            return True
+    return False
+
+
+def extract_jsons(text):
+    """
+    Extracts all valid JSON objects from a given string.
+
+    Parameters:
+        text (str): The string from which JSON objects are to be extracted.
+
+    Returns:
+        List[Dict]: A list of all extracted JSON objects.
+    """
+    text = re.sub(r"\s+", " ", text)
+    matches = re.findall(r"\{.*?\}", text)
+    parsed_jsons = []
+    for match in matches:
+        try:
+            json_object = json.loads(match)
+            parsed_jsons.append(json_object)
+        except ValueError:
+            pass
+    return parsed_jsons
+
+
+def extract_code(text):
+    """
+    Extracts all code blocks encapsulated by '```' from a given string.
+
+    Parameters:
+        text (str): The string from which Python code blocks are to be extracted.
+
+    Returns:
+        List[str]: A list of all extracted Python code blocks.
+    """
+    text = re.sub("```python", "```", text)
+    matches = re.findall(r"```(.*?)```", text, re.DOTALL)
+    parsed_codes = []
+    for match in matches:
+        parsed_codes.append(match)
+    return parsed_codes
+
+
+class AttributedDict(dict):
+    """
+    A dictionary class whose keys are automatically set as attributes of the class.
+
+    The dictionary is serializable to JSON.
+
+    Inherits from:
+        dict: Built-in dictionary class in Python.
+
+    Note:
+        This class provides attribute-style access to dictionary keys, meaning you can use dot notation
+        (like `my_dict.my_key`) in addition to the traditional bracket notation (`my_dict['my_key']`).
+    """
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+    def __setattr__(self, key, value):
+        self[key] = value
+
+    def __getattr__(self, key):
+        if key in self:
+            return self[key]
+        raise AttributeError
+
+    def __delattr__(self, key):
+        del self[key]
+
+    # check whether the key is string when adding the key
+    def __setitem__(self, key, value):
+        if not isinstance(key, str):
+            raise ValueError("The key must be a string")
+        super().__setitem__(key, value)
+
+    def update(self, *args, **kwargs):
+        for key, value in dict(*args, **kwargs).items():
+            self[key] = value

arguments.py

@@ -0,0 +1,156 @@
+import argparse
+import logging
+import os
+import sys
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="Argument parser for configuring OpenAI API and experiment settings")
+
+    # Authentication details for OpenAI API
+    parser.add_argument(
+        "--openai_key", type=str, default=None, help="API key to authenticate with OpenAI. Can be set via this argument or through the OPENAI_API_KEY environment variable."
+    )
+
+    parser.add_argument(
+        "--deployment", type=str, default=None, help="For Azure OpenAI: the deployment name to be used when calling the API."
+    )
+
+    parser.add_argument(
+        "--openai_client_type", type=str, default="openai", choices=["openai", "azure_openai"],
+        help="Specify the OpenAI client type to use: 'openai' for standard OpenAI API or 'azure_openai' for Azure-hosted OpenAI services."
+    )
+
+    parser.add_argument(
+        "--endpoint", type=str, default=None, help="For Azure OpenAI: custom endpoint to access the API. Should be in the format 'https://<your-endpoint>.openai.azure.com'."
+    )
+
+
+    parser.add_argument(
+        "--api_version", type=str, default="2023-03-15-preview", help="API version to be used for making requests. Required for Azure OpenAI clients."
+    )
+
+    # Experiment configuration
+    parser.add_argument(
+        "--ac_scoring_method", type=str, default="ranking", choices=["recommendation", "ranking"],
+        help="Specifies the scoring method used by the Area Chair (AC) to evaluate papers: 'recommendation' or 'ranking'."
+    )
+
+    parser.add_argument(
+        "--conference", type=str, default="ICLR2023", help="Conference name where the papers are being evaluated, e.g., 'ICLR2023'."
+    )
+
+    parser.add_argument(
+        "--num_reviewers_per_paper", type=int, default=3, help="The number of reviewers assigned to each paper."
+    )
+
+    parser.add_argument(
+        "--experiment_name",
+        type=str, default=None, required=False,
+        choices=[
+            "BASELINE", "benign_Rx1", "malicious_Rx1", "malicious_Rx2", "malicious_Rx3", "unknowledgeable_Rx1",
+            "knowledgeable_Rx1", "responsible_Rx1", "irresponsible_Rx1", "irresponsible_Rx2", "irresponsible_Rx3",
+            "inclusive_ACx1", "authoritarian_ACx1", "conformist_ACx1", "no_numeric_ratings"],
+        help="Specifies the name of the experiment to run. Choose from predefined experiment types based on the reviewer and AC behavior or experiment configuration."
+    )
+
+    parser.add_argument(
+        "--ignore_missing_metareviews", action="store_true", help="If set, missing metareviews are ignored, allowing the experiment to continue without them."
+    )
+
+    parser.add_argument(
+        "--overwrite", action="store_true", help="If set, existing results or output files will be overwritten without prompting."
+    )
+
+    parser.add_argument(
+        "--num_papers_per_area_chair", type=int, default=10, help="The number of papers each area chair is assigned for evaluation."
+    )
+
+    # Model configuration
+    parser.add_argument(
+        "--model_name", type=str, default="gpt-4o", choices=["gpt-4", "gpt-4o", "gpt-35-turbo"],
+        help="Specifies which GPT model to use: 'gpt-4' for the standard GPT-4 model, 'gpt-35-turbo' for a "
+             "cost-effective alternative, or 'gpt-4o' for larger context support."
+    )
+
+    # Output directories
+    parser.add_argument(
+        "--output_dir", type=str, default="outputs", help="Directory where results, logs, and outputs will be stored."
+    )
+
+    # Output directories
+    parser.add_argument(
+        "--max_num_words", type=int, default=16384, help="Maximum number of words in the paper."
+    )
+
+    parser.add_argument(
+        "--visual_dir", type=str, default="outputs/visual", help="Directory where visualization files (such as graphs and plots) will be stored."
+    )
+
+    # System configuration
+    parser.add_argument(
+        "--device", type=str, default='cuda', help="The device to be used for processing (e.g., 'cuda' for GPU acceleration or 'cpu' for standard processing)."
+    )
+
+    parser.add_argument(
+        "--data_dir", type=str, default='data', help="Directory where input data (e.g., papers) are stored."
+    )
+
+
+    parser.add_argument(
+        "--acceptance_rate", type=float, default=0.32,
+        help="Percentage of papers to accept. We use 0.32, the average acceptance rate for ICLR 2020 - 2023"
+    )
+
+    args = parser.parse_args()
+
+    # Ensure necessary directories exist
+    os.makedirs(args.visual_dir, exist_ok=True)
+    os.makedirs(args.output_dir, exist_ok=True)
+
+    # Set 'player_to_test' based on experiment name
+    if args.experiment_name is None:
+        args.player_to_test = None
+    elif "Rx" in args.experiment_name:
+        args.player_to_test = "Reviewer"
+    elif "ACx" in args.experiment_name:
+        args.player_to_test = "Area Chair"
+    elif "no_rebuttal" in args.experiment_name or "no_overall_score" in args.experiment_name:
+        args.player_to_test = "Review Mechanism"
+
+    # Sanity checks for authentication
+    print("Running sanity checks for the arguments...")
+
+    if args.openai_client_type == "openai":
+        if os.environ.get('OPENAI_API_KEY') is None:
+            assert isinstance(args.openai_key, str), ("Please specify the `--openai_key` argument OR set the "
+                                                      "OPENAI_API_KEY environment variable.")
+            raise ValueError("OpenAI key is missing.")
+
+    if args.openai_client_type == "azure_openai":
+        if os.environ.get('AZURE_OPENAI_KEY') is None:
+            assert isinstance(args.openai_key, str), ("Please specify the `--openai_key` argument OR set the "
+                                                      "AZURE_OPENAI_KEY environment variable.")
+            os.environ['AZURE_OPENAI_KEY'] = args.openai_key
+
+        if os.environ.get('AZURE_DEPLOYMENT') is None:
+            assert isinstance(args.deployment, str), ("Please specify the `--deployment` argument OR set the "
+                                                      "AZURE_DEPLOYMENT environment variable.")
+            os.environ['AZURE_DEPLOYMENT'] = args.deployment
+
+        if os.environ.get('AZURE_ENDPOINT') is None:
+            assert isinstance(args.endpoint, str), ("Please specify the `--endpoint` argument OR set the "
+                                                    "AZURE_ENDPOINT environment variable.")
+            endpoint = args.endpoint
+        else:
+            endpoint = os.environ.get('AZURE_ENDPOINT')
+
+        if not endpoint.startswith("https://"):
+            endpoint = f"https://{endpoint}.openai.azure.com"
+        os.environ['AZURE_ENDPOINT'] = endpoint
+
+        if os.environ.get('OPENAI_API_VERSION') is None:
+            assert isinstance(args.api_version, str), ("Please specify the `--api_version` argument OR set the "
+                                                       "OPENAI_API_VERSION environment variable.")
+            os.environ['OPENAI_API_VERSION'] = args.api_version
+
+    return args

const.py

@@ -0,0 +1,100 @@
+
+
+"""
+Note
+- ICLR 2021 has a category for "Significant-concerns"
+- ICLR 2023 categories the papers as "Accept-notable-top-5", "Accept-notable-top-25", "Accept-poster", and "Reject"
+"""
+PAPER_DECISIONS = ["Reject","Accept-oral", "Accept-spotlight", "Accept-poster",]
+PAPER_DECISIONS_ICLR2019 = ["Accept-oral", "Accept-poster", "Reject"]
+
+AREA_CHAIR_TYPES = ['inclusive', 'conformist', 'authoritarian', 'BASELINE']
+
+# These are papers that contain potentially sensitive content. GPT-4 refused to generate reviews for these papers.
+FILTERED_PAPER_IDS = {
+    "ICLR2020": [],
+    "ICLR2021": [],
+    "ICLR2022": [186, 200, 270],
+    "ICLR2023": []
+}
+
+ALL_REVIEW_PHASES = ["reviewer_write_reviews", "author_reviewer_discussion", "reviewer_ac_discussion", "ac_discussion"]
+
+
+EXPERIMENT_NAME2REVIEWER_TYPES = {
+    "BASELINE": "BASELINE",
+    "knowledgeable_Rx1": "knowledgeable",
+    "unknowledgeable_Rx1": "unknowledgeable",
+    "irresponsible_Rx1": "irresponsible",
+    "irresponsible_Rx2": "irresponsible",
+    "irresponsible_Rx3": "irresponsible",
+    "responsible_Rx1": "responsible",
+    "malicious_Rx1": "malicious",
+    "malicious_Rx2": "malicious",
+    "malicious_Rx3": "malicious",
+    "benign_Rx1": "benign",
+    "inclusive_ACx1": "BASELINE",
+    "authoritarian_ACx1": "BASELINE",
+    "conformist_ACx1": "BASELINE",
+    "authors_are_famous_Rx1": "authors_are_famous",
+    "authors_are_famous_Rx2": "authors_are_famous",
+    "authors_are_famous_Rx3": "authors_are_famous",
+    "authors_are_famous_Rx1_no_rebuttal": "authors_are_famous",
+    "authors_are_famous_Rx2_no_rebuttal": "authors_are_famous",
+    "authors_are_famous_Rx3_no_rebuttal": "authors_are_famous",
+    "no_rebuttal": "BASELINE",
+    "no_overall_score": "BASELINE",
+}
+
+
+year2paper_ids = {
+    "ICLR2018": [45, 47, 59, 76, 229, 254, 372, 415, 447, 517, 543, 544, 562, 596, 615, 639] +
+                    [1, 2, 7, 10, 16, 26, 33, 51, 60, 61, 65,
+                    67, 69, 72, 73, 77, 84, 88, 94, 99, 104,
+                    117, 121, 124, 131, 132, 134, 136, 143, 147, 148, 149, 155, 162, 164, 166, 168, 169, 171,
+                     175, 178, 179, 189, 196, 201, 203, 204, 205] +
+                [3, 4, 6, 8, 9, 11, 12, 13, 15, 17, 18, 19, 20, 21, 24, 25, 27, 28, 30, 31, 32, 34, 36, 37, 39, 40,
+                 41, 42, 43, 44, 46, 52, 53, 54, 55, 56, 58, 63, 66, 68, 71, 74, 75, 78, 80, 83, 85,  87, 89,
+                 91, 92, 93, 95, 96, 97, 100, 101, 102, 103, 105, 106, 107, 108, 109, 110, 111, 113, 114, 115,
+                 116, 118, 120, 122, 123, 125, 127, 128, 129, 133, 135, 138, 141, 142, 144, 153, 154, 156, 157, 158,
+                 159, 161, 163, 170, 172, 173, 174, 176, 177, 180, 181, 182, 184, 185, 186, 187, 190, 191, 193, 194,
+                 197, 200, 206, 207, 209, 211, 213, 214, 218, 219, 221, 222, 223, 225, 226, 230, 234, 237, 238, 241,
+                 243, 244, 247, 248, 253, 255, 256, 257, 258, 259, 266, 268, 271, 272, 273, 275, 276, 278, 283,
+                 286],
+
+
+    "ICLR2019": [1, 26, 119, 220, 231, 507, 563, 566, 574, 632, 654, 709, 734, 780, 835, 917] + [4, 27, 33, 39, 40,
+                                                                                                 51, 57, 67, 70, 72,
+                                                                                                 73, 76, 77, 82, 87,
+                                                                                                 98, 99, 100, 106,
+                                                                                                 108, 109, 110, 111,
+                                                                                                 113, 114, 116, 123,
+                                                                                                 129, 130, 143, 146,
+                                                                                                 147, 150, 155, 177,
+                                                                                                 184, 187, 190, 194,
+                                                                                                 201, 202, 203, 205,
+                                                                                                 211, 213, 222, 237,
+                                                                                                 238] + [2, 3, 6, 8,
+                                                                                                         9, 11, 12,
+                                                                                                         13, 14, 15,
+                                                                                                         16, 17, 18,
+                                                                                                         19, 20, 21,
+                                                                                                         22, 23, 24,
+                                                                                                         28, 29, 32,
+                                                                                                         35, 36, 37,
+                                                                                                         38, 41, 42,
+                                                                                                         43, 44, 45, 46, 47, 48, 49, 50, 52, 54, 55, 58, 59, 60, 61, 62, 63, 65, 66, 68, 69, 71, 74, 75, 78, 79, 80, 83, 84, 85, 86, 89, 90, 91, 92, 93, 94, 95, 96, 97, 101, 102, 104, 105, 107, 112, 115, 117, 118, 120, 122, 124, 125, 127, 128, 131, 132, 133, 134, 135, 136, 137, 139, 140, 141, 142, 144, 145, 148, 149, 152, 153, 154, 158, 160, 161, 162, 163, 164, 165, 167, 168, 171, 172, 173, 174, 178, 179, 180, 181, 182, 185, 186, 189, 191, 192, 193, 195, 196, 197, 198, 204, 206, 208, 209, 210, 214, 216, 217, 218, 221, 223, 224, 225, 226, 228, 229, 230, 233],
+
+
+
+    "ICLR2020": [2, 3, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 21, 23, 27, 28, 31, 32, 33, 34, 35, 37, 38,
+                 41, 42, 43, 46, 48, 49, 52, 56, 58, 60, 62, 63, 66, 67, 68, 74, 75, 76, 79, 80, 82, 83, 84, 85, 86,
+                 87, 90, 94, 97, 101, 105, 106, 107, 109, 110, 111, 115, 116, 118, 119, 120, 121, 122, 123, 124, 126,
+                 1, 16, 18, 22, 24, 29, 40, 45, 50, 53, 55, 57, 61, 70, 73, 77, 91, 93, 95, 103, 108, 112, 113, 117,
+                 125, 132, 5, 44, 47, 54, 71, 88, 69, 78, 102, 221],
+    "ICLR2021": [140, 218, 294, 332, 362, 420, 1, 5, 12, 20, 28, 52, 67, 69, 75, 102, 103, 110, 126, 135, 138, 147, 149, 151, 160, 170, 174, 181, 182, 190, 44, 3, 4, 10, 11, 14, 17, 18, 21, 22, 24, 27, 33, 36, 38, 45, 47, 49, 59, 70, 72, 73, 78, 79, 82, 84, 86, 88, 89, 91, 92, 98, 100, 101, 104, 105, 106, 107, 108, 111, 114, 120, 123, 124, 125, 130, 131, 133, 137, 141, 142, 143, 154, 155, 156, 157, 159, 161, 162, 164, 166, 167, 168, 171, 172, 176, 193, 194, 81, 94, 95, 146, 177, 179, 184, 186],
+    "ICLR2022": [86, 154, 208, 222, 224, 284, 9, 10, 11, 12, 14, 25, 30, 31, 39, 42, 45, 56, 68, 73, 80, 88, 89, 90, 96, 101, 102, 104, 109, 1, 4, 6, 27, 36, 43, 47, 61, 62, 63, 65, 67, 69, 81, 82, 95, 98, 99, 100, 103, 105, 106, 108, 115, 120, 121, 122, 130, 134, 142, 143, 144, 145, 152, 153, 157, 159, 168, 173, 174, 175, 176, 179, 180, 186, 187, 193, 194, 197, 200, 201, 205, 210, 216, 226, 229, 233, 234, 235, 236, 239, 248, 261, 262, 263, 264, 269, 270, 271, 112, 113, 34, 64, 158, 172, 277, 280, 283, 286],
+    "ICLR2023": [210, 219, 1759, 1774, 9, 11, 12, 33, 54, 55, 61, 70, 79, 86, 88, 90, 97, 116, 128, 129, 143, 152, 160, 168, 174, 177, 181, 193, 1647, 1651, 1666, 1670, 1673, 1675, 1677, 1678, 1680, 1683, 1692, 1698, 1703, 1709, 1716, 1720, 1723, 1727, 1728, 1742, 1743, 1752, 1754, 1760, 113, 156, 214, 220, 317, 318, 1657, 1686, 1740, 1762, 1783, 1817, 2, 3, 6, 8, 17, 18, 24, 25, 29, 30, 45, 62, 77, 80, 82, 84, 89, 96, 104, 105, 107, 108, 118, 119, 120, 122, 130, 131, 133, 139, 141, 145, 146, 149, 150, 151, 153, 158, 161, 163, 164, 169, 175, 178, 179, 198, 200, 206, 207, 211, 212, 225, 226, 231, 235, 236, 237, 245, 246, 249, 252, 253, 255, 257, 258, 259, 264, 265, 266, 275, 1645, 1649, 1655, 1658, 1663, 1664, 1665, 1672, 1679, 1682, 1685, 1695, 1697, 1701, 1704, 1706, 1708, 1710, 1712, 1713, 1715, 1722, 1726, 1729, 1731, 1734, 1736, 1738, 1739, 1741, 1744, 1745, 1749, 1750, 1755, 1756, 1758, 1761, 1764, 1767, 1772, 1773, 1778, 1779, 1780, 1786, 1788, 1790, 1791, 1795, 1796, 1797, 1800, 1802, 1803, 1805, 1810, 1812, 1813, 1821, 1822, 1827, 1829, 1833, 1840, 1845, 1851, 1856],
+    "ICLR2024": [39, 247, 289, 400, 489, 742, 749] + [62, 78, 159, 161, 170, 192, 198, 215, 219, 335, 344, 386, 427, 432, 448, 451, 461, 472, 485, 536, 546, 559, 573, 577, 597] + [5, 9, 11, 19, 20, 30, 31, 32, 40, 49, 52, 53, 54, 56, 61, 66, 67, 73, 74, 77, 85, 87, 100, 104, 114, 116, 124, 130, 133, 138, 145, 151, 153, 156, 165, 166, 172, 181, 183, 187, 195, 204, 212, 221, 224, 230, 237, 243, 248, 257, 258, 259, 263, 272, 278, 287, 288, 291, 292, 298, 300, 302, 304, 306, 308, 318, 320, 321, 324, 325, 326, 327, 331, 332, 334, 336, 338, 340, 345, 349, 350, 356, 357, 358, 360] + [1, 2, 12, 14, 24, 26, 33, 35, 36, 41, 42, 44, 50, 51, 55, 57, 59, 70, 72, 75, 76, 81, 89, 90, 93,
+                  94, 97, 99, 101, 105, 110, 111, 112, 117, 119, 120, 125, 128, 129, 131, 134, 135, 140, 148, 150, 157, 158, 163, 167, 173, 175, 177, 182, 185, 186, 188, 189, 197, 202, 207, 209, 210, 214, 216, 226, 231, 234, 236, 238, 239, 241, 244, 245, 249, 260, 262, 264, 265, 271, 276, 277, 279, 281, 282, 284, 286, 290, 294, 295, 301, 303, 307, 309, 313, 315, 319, 322, 333, 337, 339, 342, 354, 363, 364, 369, 373, 374, 375, 377, 378, 381, 382, 385, 388, 398, 399, 401, 407, 412, 413, 415, 416, 417, 420, 421, 422, 426, 428, 436, 437, 444, 446, 449, 453, 454, 463, 464, 469, 478, 480, 487, 490, 496, 498, 501, 502, 504, 506, 513, 516, 517, 518, 520, 521, 523, 524, 525, 537, 541, 545, 551, 552, 554, 555, 558, 562, 563, 574, 575, 579, 581, 584, 588, 595, 596, 598, 607, 608, 615, 622, 624, 625, 627, 629, 630, 634, 636, 641, 645, 647, 648, 651, 652, 654, 655, 662, 667, 668, 671, 672, 673, 681, 682, 685, 689, 690, 691, 697, 698, 701]
+}

docs/devdoc/design.md

@@ -0,0 +1,39 @@
+# Key Design Choices
+In this document, we will discuss the key concepts and design choices of ChatArena.
+We expect this will be helpful particularly for developers who want to contribute to ChatArena or build their own environments.
+
+## Agent Environment Cycle
+ChatArena in general follows the design principle of openAI gym [1] and pettingzoo [2]. Any agent will interact with the environment and other agents through the agent environment cycle.
+For every single cycle,
+1. the agent observes the environment
+2. the agent output an action
+3. the environment makes a state transition given the action
+
+As an optional feature, in each cycle, the environment can also compute a scalar reward for every single agent, along with a terminal signal for the environment.
+
+[1] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba: OpenAI Gym. CoRR abs/1606.01540 (2016)
+
+[2] Justin K. Terry, Benjamin Black, Nathaniel Grammel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis S. Santos, Clemens Dieffendahl, Caroline Horsch, Rodrigo Perez-Vicente, Niall L. Williams, Yashas Lokesh, Praveen Ravi: PettingZoo: Gym for Multi-Agent Reinforcement Learning. NeurIPS 2021: 15032-15043
+
+### Actions
+
+In the current version of ChatArena, all the actions are represented as plain text. More structured text outputs, like json or code, can be generated by prompting the LLM to do so.
+We provide simple utilities to extract json and code (with markdown syntax), which should cover common use cases but can break for intentionally crafted edge cases.
+
+### Observations
+
+A observation is a list of messages with sender and content. Then sender can be any agent in the environment or the built-in moderator of the environment. The content is again plain text.
+
+## Message Pool and Visibility Control
+
+In ChatArena, agents cannot directly talk to each other but exchange information with a [message pool](https://github.com/chatarena/chatarena/blob/main/chatarena/message.py) as a proxy. The message pool is a utility abstraction that can serve as a part of the game state.
+
+When an agent takes an action, a message can be created and appended to the message pool. In the message pool, each message will have a receiver, which can be decided by the environment dynamics (game rules) or by the agent itself. The environment itself can also create messages under the name of the moderator which can provide other state information or extra instructions given the current state.
+
+To render an observation, the message pool will collect all the messages that are visible to the agent and return a list of these messages.
+
+In particular, some of the environments require parallel moves, say, rock-paper-scissors, where the agent shouldn’t see the moves of other agents in the same turn. Such a mechanism is also implemented in the message pool. One can specify the “current turn” or the message of the “current turns” and turns after will be ignored.
+
+## Intelligence Backends
+
+In ChatArena, each agent will usually be powered by a language backend. These backends can be LLM APIs (say, from [OpenAI](https://github.com/chatarena/chatarena/blob/main/chatarena/backends/openai.py), [Anthropic](https://github.com/chatarena/chatarena/blob/main/chatarena/backends/anthropic.py) or [Cohere](https://github.com/chatarena/chatarena/blob/main/chatarena/backends/cohere.py)), [local LLM](https://github.com/chatarena/chatarena/blob/main/chatarena/backends/hf_transformers.py) or just [humans](https://github.com/chatarena/chatarena/blob/main/chatarena/backends/human.py) behind a user interface. In [backends](https://github.com/chatarena/chatarena/tree/main/chatarena/backends), we render the observations (list of messages) into the required formats for the downstream models. And the returned text will be the agent’s action [by default](https://github.com/chatarena/chatarena/blob/55c9e6ee4e09d72905eceb0a0e09e93a4179ca39/chatarena/agent.py#L28).

docs/devdoc/mainloop.md

@@ -0,0 +1,62 @@
+### Step 1: Define Multiple Players with LLM Backend
+
+```python
+from agentreview.agent import Player
+from agentreview.backends import OpenAIChat
+
+# Describe the environment (which is shared by all players)
+environment_description = "It is in a university classroom ..."
+
+# A "Professor" player
+player1 = Player(name="Professor", backend=OpenAIChat(),
+                 role_desc="You are a professor in ...",
+                 global_prompt=environment_description)
+# A "Student" player
+player2 = Player(name="Student", backend=OpenAIChat(),
+                 role_desc="You are a student who is interested in ...",
+                 global_prompt=environment_description)
+# A "Teaching Assistant" player
+player3 = Player(name="Teaching assistant", backend=OpenAIChat(),
+                 role_desc="You are a teaching assistant of the module ...",
+                 global_prompt=environment_description)
+```
+
+### Step 2: Create a Language Game Environment
+
+You can also create a language model-driven environment and add it to the ChatArena:
+
+```python
+from agentreview.environments.conversation import Conversation
+
+env = Conversation(player_names=[p.name for p in [player1, player2, player3]])
+```
+
+### Step 3: Run the Language Game using Arena
+
+`Arena` is a utility class to help you run language games:
+
+```python
+from agentreview.arena import Arena
+
+arena = Arena(players=[player1, player2, player3],
+              environment=env, global_prompt=environment_description)
+# Run the game for 10 steps
+arena.run(num_steps=10)
+
+# Alternatively, you can run your own main loop
+for _ in range(10):
+    arena.step()
+    # Your code goes here ...
+```
+
+You can easily save your gameplay history to file:
+
+```python
+arena.save_history(path=...)
+```
+
+and save your game config to file:
+
+```python
+arena.save_config(path=...)
+```

docs/devdoc/moderated.md

@@ -0,0 +1,16 @@
+### `ModeratedConversation`: a LLM-driven Environment
+
+We support a more advanced environment called `ModeratedConversation` that allows you to **control the game dynamics
+using an LLM**.
+The moderator is a special player that controls the game state transition and determines when the game ends.
+For example, you can define a moderator that tracks the board status of a board game and ends the game when a player
+wins.
+You can try out our Tic-tac-toe and Rock-paper-scissors games to get a sense of how it works:
+
+```python
+# Tic-tac-toe example
+Arena.from_config("examples/tic-tac-toe.json").launch_cli()
+
+# Rock-paper-scissors example
+Arena.from_config("examples/rock-paper-scissors.json").launch_cli()
+```

docs/tutorials/create_your_environment.md

@@ -0,0 +1,90 @@
+# How to create your custom environments
+
+As an example to demonstrate how to develop your own environment, we develop a language
+game based on [The Chameleon](https://bigpotato.co.uk/blogs/blog/how-to-play-the-chameleon-instructions).
+The example code is available [here](../../agentreview/environments/chameleon.py).
+
+**Here are the detailed steps to develop a custom environment class**
+
+1. **Define the class**: Start by defining the class and inherit from a suitable base class (e.g., `Environment`). In
+   this case, the custom class `Chameleon` inherits from the `Environment` base class.
+
+```python
+class Chameleon(Environment):
+    type_name = "chameleon"
+```
+
+The `type_name` is required and it is used by the [`ENV_REGISTRY`](chatarena/environments/__init__.py#L13) to identify
+the class when loading the class
+from a config file.
+
+Make sure you add the class to [`ALL_ENVIRONMENTS`](chatarena/environments/__init__.py#L17)
+in `environments/__init__.py` so that it can be detected.
+
+2. **Initialize the class**: Define the `__init__` method to initialize the class attributes, such as player names, game
+   state, and any other necessary variables.
+
+```python
+def __init__(self, player_names: List[str], topic_codes: Dict[str, List[str]] = None, **kwargs):
+    super().__init__(player_names=player_names, ..., **kwargs)
+    ...
+
+    # The "state" of the environment is maintained by the message pool
+    self.message_pool = MessagePool()
+    ...
+```
+
+3. **Implement game mechanics**: Write methods that define the game mechanics, such as giving clues, voting, and
+   guessing the secret word. In the `Chameleon` class, these mechanics are implemented in the `step` method.
+
+```python
+def step(self, player_name: str, action: str) -> TimeStep:
+    ...
+```
+
+4. **Handle game states and rewards**: Implement methods to manage game states, such as resetting the environment,
+   getting
+   observations, checking if the game has reached a terminal state, and giving rewards to players.
+
+```python
+def reset(self):
+    ...
+
+
+def get_observation(self, player_name=None) -> List[Message]:
+    ...
+
+
+def is_terminal(self) -> bool:
+    ...
+
+
+def get_rewards(self, ...) -> Dict[str, float]:
+    ...
+```
+
+5. **Develop your role description prompts for the players**: Now that you have defined the game mechanics, you can
+   develop the role description prompts for the players. These prompts are used to guide the LLM-powered players to play
+   the game
+   correctly. You can use the CLI for this purpose. For example, you can run the following code to launch the CLI:
+
+```python
+alice = Player(name="Alice", backend=OpenAIChat(), role_desc="Write your prompt here")
+bob = Player(name="Bob", backend=OpenAIChat(), role_desc="Write your prompt here")
+env = Chameleon(player_names=["Alice", "Bob"], topic_codes=...)
+arena = Arena(players=[alice, bob], environment=env).launch_cli()
+```
+
+Once you are happy with you prompts, you can save them to a config file for future use or sharing.
+
+```python
+arena.save_config(path=...)
+```
+
+Another option is using the Web UI. You can run the following code to launch the Web UI:
+
+```bash
+gradio app.py
+```
+
+and select your custom environment from the dropdown menu.

requirements.txt

@@ -0,0 +1,19 @@
+
+colorama
+llama_index
+matplotlib
+numpy
+openreview_py
+pandas
+prompt_toolkit
+requests
+rich
+setuptools
+tenacity
+tiktoken
+tqdm
+transformers
+tenacity
+openai
+gradio
+

review_content_analysis/analysis.py

@@ -0,0 +1,477 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
+import os
+import re
+import time
+import json
+import matplotlib.pyplot as plt
+from openai import OpenAI
+import multiprocessing
+
+FONT_SIZE = 20
+
+COLORS = ['#26547c', '#06d6a0', '#ef476f', '#ffd166']
+
+openai_api_key = os.getenv("OPENAI_KEY")
+# print(openai.api_key)
+
+base_dir = '/home/v-qinlinzhao/agent4reviews/simulated_review/reviews'
+save_base_dir = '/home/v-qinlinzhao/agent4reviews/simulated_review/classified_reason/'
+
+with open('iter_prompt.txt', 'r') as f:
+    iter_prompt = f.read()
+    
+with open('classification_prompt.txt', 'r') as f:
+    classification_prompt = f.read()
+
+with open('reason_library.txt', 'r') as f:
+    reason_library = f.read()
+
+def get_gpt_response(prompt):
+    client = OpenAI(api_key=openai_api_key)
+    messages = [{'role': 'user', 'content': prompt}]
+    completion = client.chat.completions.create(
+            model="gpt-4-1106-preview",
+            messages=messages,
+            temperature=0.7,
+            max_tokens=2000,
+        )
+
+    response = completion.choices[0].message.content
+    response = response.strip()
+    
+    # time.sleep(5)
+    return response
+
+def extract_review_from_real_data():
+    base_dir = '/home/v-qinlinzhao/agent4reviews/real_review/original_data'
+    result_dir = '/home/v-qinlinzhao/agent4reviews/real_review/extracted_real_review/'
+    # 目录为 ICLR202X/notes/xxx.json
+    # 将其中所有的json文件的review提取处理
+    for root, dirs, files in os.walk(base_dir):
+        for file in files:
+            if file.endswith('.json'):
+                with open(os.path.join(root, file), 'r') as f:
+                    data = json.load(f)
+                    reviews = []
+                    data = data['details']['replies']
+                    id = []
+                    for d in data:
+                        if d['id'] not in id:
+                            id.append(d['id'])
+                            # 2020-2021
+                            if 'content' in d and 'review' in d['content']:
+                                reviews.append(d['content']['review'])
+                            # 2022
+                            if 'content' in d and 'main_review' in d['content']:
+                                reviews.append(d['content']['main_review'])
+                            # 2023
+                            if 'content' in d and 'strength_and_weaknesses' in d['content']:
+                                reviews.append(d['content']['strength_and_weaknesses'])
+                                
+                    # 将每个review分别存入到json文件中，命名格式为 {当前文件名}_{序号}.json
+                    # 同时保持每个文件在原目录下相对路径    
+                    relative_dir = os.path.relpath(root, base_dir)
+                    result_file_dir = os.path.join(result_dir, relative_dir)
+                    os.makedirs(result_file_dir, exist_ok=True) 
+                       
+                    file_base_name = os.path.splitext(file)[0]
+                    
+                    for i, review in enumerate(reviews): 
+                        result_file_name = f"{file_base_name}_{i}.json"
+                        result_file_path = os.path.join(result_file_dir, result_file_name)
+                        
+                        with open(result_file_path, 'w') as result_file:
+                            json.dump({"review": review}, result_file, ensure_ascii=False, indent=4)
+
+def extract_meta_review_from_simulated_data():
+    base_dir = '/home/v-qinlinzhao/agent4reviews/simulated_review/full_paper_discussion'
+    result_dir = '/home/v-qinlinzhao/agent4reviews/simulated_review/meta_review/'
+    # 目录为 ICLR202X/notes/xxx.json
+    # 将其中所有的json文件的review提取处理
+    for root, dirs, files in os.walk(base_dir):
+        for file in files:
+            if file.endswith('.json'):
+                with open(os.path.join(root, file), 'r') as f:
+                    data = json.load(f)
+                    # review在data['messages']中最后一个元素中的"content"中
+                    review = data['messages'][-1]['content']
+                # write review into file, keep the abstract path 
+                relative_dir = os.path.relpath(root, base_dir)
+                result_file_dir = os.path.join(result_dir, relative_dir)
+                os.makedirs(result_file_dir, exist_ok=True)
+                result_file_path = os.path.join(result_file_dir, file)
+                with open(result_file_path, 'w') as result_file:
+                    json.dump({"meta_review": review}, result_file, ensure_ascii=False, indent=4)
+                    
+# Select 1% of the data randomly, let GPT-4 summarize the reasons, and add them to the reason library if there are reasons that do not exist
+def construct_reason_library():
+    
+    base_dir = '/home/v-qinlinzhao/agent4reviews/paper_review_and_rebuttal/selected_files/'
+    
+    json_files = []
+    for root, dirs, files in os.walk(base_dir):
+        for file in files:
+            if file.endswith('.json'):
+                json_files.append(os.path.join(root, file))
+
+    for file in json_files:
+        with open(file, 'r') as f:
+            data = json.load(f)
+            review = data['review']
+            prompt = iter_prompt.format(review=review, 
+                                        reason_library=reason_library)
+            ans = get_gpt_response(prompt)
+            print(ans)
+
+def analyze_reason_in_batch(json_files):
+    
+    for file in json_files:
+        with open(file, 'r') as f:
+            data = json.load(f)
+            review = data['review']
+            prompt = classification_prompt.format(review=review)
+            res = get_gpt_response(prompt)
+            
+            # 解析res的输出，将accept和reject的原因分别提取出来，写成json格式
+            # 依据该字符串分别抽取Accept和Reject的原因
+            reason_dict = {}
+            if 'Reject' in res:
+                accept_reason = re.search(r"Accept: (.+?);", res)
+            else:
+                accept_reason = re.search(r"Accept: (.+)", res)
+                
+            reject_reason = re.search(r"Reject: (.+)", res)
+            # print(reject_reason)
+            if accept_reason:
+                accept_reason = accept_reason.group(1).split(',')
+                reason_dict['accept'] = []
+                for r in accept_reason:
+                    r = r.strip()
+                    if r in ['1', '2', '3', '4', '5']:
+                        reason_dict['accept'].append(r)
+            if reject_reason:
+                reject_reason = reject_reason.group(1).split(',')
+                reason_dict['reject'] = []
+                for r in reject_reason:
+                    r = r.strip()
+                    if r in ['1', '2', '3', '4', '5', '6', '7']:
+                        reason_dict['reject'].append(r)
+    
+            # print(res)
+            relative_path = os.path.relpath(file, base_dir)
+            save_path = os.path.join(save_base_dir, relative_path)
+            save_dir = os.path.dirname(save_path)
+       
+            # 首先找到原来目录的目录结构，然后在save_dir中按照该目录保存结果保存结果
+            if not os.path.exists(save_dir):
+                os.makedirs(save_dir)
+            with open(save_path, 'w') as f:
+                json.dump(reason_dict, f, indent=4)
+
+def convert_txt_to_json():
+    base_dir = '/home/v-qinlinzhao/agent4reviews/simulated_review/classified_meta_review_reason'
+    reason_count = {}
+    reason_total_count = {'accept': {}, 'reject': {}}
+
+    def process_directory(path, reason_dict):
+        # 迭代path下的内容
+        for item in os.listdir(path):
+            item_path = os.path.join(path, item)
+            if os.path.isdir(item_path):
+                # 如果是目录，递归处理
+                reason_dict[item] = {}
+                process_directory(item_path, reason_dict[item])
+            elif item.endswith('.txt'):
+                # 去除txt后缀
+                item_name = item.replace('.txt', '')
+                reason_dict[item_name] = {'accept': {}, 'reject': {}}
+                # 如果是txt文件，处理文件内容
+                with open(item_path, 'r') as f:
+                    content = f.read()
+                    # "Accept: 1,2,3; Reject: 3,4,7"
+                    # 依据该字符串分别抽取Accept和Reject的原因
+                    if 'Reject' in content:
+                        accept_reason = re.search(r"Accept: (.+?);", content)
+                    else:
+                        accept_reason = re.search(r"Accept: (.+)", content)
+                    reject_reason = re.search(r"Reject: (.+)", content)
+                    # print(reject_reason)
+                    if accept_reason:
+                        accept_reason = accept_reason.group(1).split(',')
+                        reason_dict[item_name]['accept'] = []
+                        for r in accept_reason:
+                            r = r.strip()
+                            if r in ['1', '2', '3', '4', '5']:
+                                if r not in reason_total_count['accept']:
+                                    reason_total_count['accept'][r] = 0
+                                reason_total_count['accept'][r] += 1
+                                reason_dict[item_name]['accept'].append(r)
+                    if reject_reason:
+                        reject_reason = reject_reason.group(1).split(',')
+                        reason_dict[item_name]['reject'] = []
+                        for r in reject_reason:
+                            r = r.strip()
+                            if r in ['1', '2', '3', '4', '5', '6', '7']:
+                                if r not in reason_total_count['reject']:
+                                    reason_total_count['reject'][r] = 0
+                                reason_total_count['reject'][r] += 1
+                                reason_dict[item_name]['reject'].append(r)
+
+    process_directory(base_dir, reason_count)
+
+    # 将统计结果写入文件
+    with open('reason.json', 'w') as f:
+        json.dump(reason_count, f, indent=4)
+    
+    # 计算accept 和 reject中每一类原因的占比
+    # reason_percentage = {'accept': {}, 'reject': {}}
+    # for key, value in reason_total_count.items():
+    #     total = sum(value.values())
+    #     for k, v in value.items():
+    #         reason_percentage[key][k] = v / total
+        
+    # with open('reason_count.json', 'w') as f:
+    #     json.dump(reason_total_count, f, indent=4)
+        
+    # with open('reason_percentage.json', 'w') as f:
+    #     json.dump(reason_percentage, f, indent=4)
+
+def count_reasons():
+    with open('../reason_result/reason.json', 'r') as f:
+        reason_count = json.load(f)
+
+    count = {}
+    for year, year_dict in reason_count.items():
+        count[year] = {}
+        for model, model_dict in year_dict.items():
+            count[year][model] = {}
+            for type, type_dict in model_dict.items():
+                count[year][model][type] = {}
+                count[year][model][type]['accept'] = {}
+                count[year][model][type]['reject'] = {}
+                # 只在type层面做统计就好了
+                for paper_id, paper_id_dict in type_dict.items():
+                    for review_id, review_id_dict in paper_id_dict.items():
+                        print(year, model, type, paper_id, review_id, review_id_dict)
+                        # {'accept': {'1': 1, '2': 1, '5': 1}, 'reject': {'3': 1, '4': 1, '5': 1, '7': 1}}
+                        if 'accept' in review_id_dict:
+                            for accept_reason in review_id_dict['accept']:
+                                if accept_reason not in count[year][model][type]['accept']  \
+                                                and accept_reason in ['1', '2', '3', '4', '5']:
+                                    count[year][model][type]['accept'][accept_reason] = 0
+                                count[year][model][type]['accept'][accept_reason] += 1
+                        if 'reject' in review_id_dict: 
+                            for reject_reason in review_id_dict['reject']:
+                                if reject_reason not in count[year][model][type]['reject'] \
+                                                and reject_reason in ['1', '2', '3', '4', '5', '6', '7']:
+                                    count[year][model][type]['reject'][reject_reason] = 0
+                                count[year][model][type]['reject'][reject_reason] += 1   
+
+    with open('reason_count.json', 'w') as f:
+        json.dump(count, f, indent=4)
+
+def calcu_reason_percentage_every_year():
+    with open('../reason_result/reason_count.json', 'r') as f:
+        reason_count = json.load(f)
+
+    distribution = {}
+    for year, year_dict in reason_count.items():
+        distribution[year] = {}
+        for model, model_dict in year_dict.items():
+            distribution[year][model] = {}
+            for type, type_dict in model_dict.items():
+                distribution[year][model][type] = {}
+                distribution[year][model][type]['accept'] = {}
+                distribution[year][model][type]['reject'] = {}
+                # 统计百分比，先将accept下面的count加起来，然后得到每个百分比
+                accept_sum = sum(type_dict['accept'].values())
+                for reason, count in type_dict['accept'].items():
+                    distribution[year][model][type]['accept'][reason] = count / accept_sum
+                reject_sum = sum(type_dict['reject'].values())
+                for reason, count in type_dict['reject'].items():
+                    distribution[year][model][type]['reject'][reason] = count / reject_sum
+    
+    with open('reason_percentage.json', 'w') as f:
+        json.dump(distribution, f, indent=4)
+ 
+def calcu_reason_percentage():
+    # 以每种类别为单位，计算每种类别下的accept和reject的百分比
+    with open('../reason_result/reason_count.json', 'r') as f:
+        reason_count = json.load(f)
+        
+    count_dict = {}
+    for year, year_dict in reason_count.items():
+        for model, model_dict in year_dict.items():
+            for type, type_dict in model_dict.items():
+                count_dict[type] = {'accept': {}, 'reject': {}}
+                # 得到所有year和model的accept和reject的count
+                accept_count = type_dict['accept']
+                reject_count = type_dict['reject']
+                # 将accept中每一类原因进行累加
+                for reason, count in accept_count.items():
+                    if reason not in count_dict[type]['accept']:
+                        count_dict[type]['accept'][reason] = 0
+                    count_dict[type]['accept'][reason] += count
+                for reason, count in reject_count.items():
+                    if reason not in count_dict[type]['reject']:
+                        count_dict[type]['reject'][reason] = 0
+                    count_dict[type]['reject'][reason] += count
+    # 计算count_dict中accept和reject其中原因的百分比
+    reason_percentage = {}
+    for type, type_dict in count_dict.items():
+        reason_percentage[type] = {'accept': {}, 'reject': {}}
+        accept_sum = sum(type_dict['accept'].values())
+        for reason, count in type_dict['accept'].items():
+            reason_percentage[type]['accept'][reason] = count / accept_sum
+        reject_sum = sum(type_dict['reject'].values())
+        for reason, count in type_dict['reject'].items():
+            reason_percentage[type]['reject'][reason] = count / reject_sum
+    
+    with open('reason_percentage.json', 'w') as f:
+        json.dump(reason_percentage, f, indent=4)
+
+def draw_bar_chart(accept_or_reject, ax, type, name1, name2):
+    # accept_or_reject = 'accept'
+    
+    x = {
+        "accept": ['Novelty', 'Significance', 'Theoretical', 'Clarity', 'Future'],
+        "reject": ['Novelty', 'Theoretical', 'Validation', 'Practicality', 'Limitations', 'Presentation', 'Related Work']
+    }
+    x_range = range(1, len(x[accept_or_reject])+1)
+    
+    # 画出每一年的type1 和 type2两种type的比例图
+    with open('../reason_result/reason_percentage.json', 'r') as f:
+        reason_percentage = json.load(f)
+    
+        # 取出其中的type1和type2两种type
+        type1 = reason_percentage[name1][accept_or_reject]
+        type2 = reason_percentage[name2][accept_or_reject]
+        
+        # 按照key排序
+        type1 = dict(sorted(type1.items(), key=lambda x: int(x[0])))
+        type2 = dict(sorted(type2.items(), key=lambda x: int(x[0])))
+        # dict中key应该是1-7，如果有的Key没有，就加上这个key，value设置为0
+        for i in x_range:
+            if str(i) not in type1:
+                type1[str(i)] = 0
+            if str(i) not in type2:
+                type2[str(i)] = 0
+        
+        width = 0.35  # 柱子的宽度
+        
+        # fig, ax = plt.subplots()
+        ax.bar([i - width/2 for i in x_range], type1.values(), width, label=name1, color=COLORS[0], alpha=0.3)
+        ax.bar([i + width/2 for i in x_range], type2.values(), width, label=name2, color=COLORS[1], alpha=0.3)
+        
+        ax.legend()
+        ax.set_xlabel('Reason', fontsize=FONT_SIZE)
+        # ax.set_ylabel('Percentage', fontsize=FONT_SIZE)
+        ax.set_title(type, fontsize=FONT_SIZE)
+        ax.set_xticks(x_range)  # 设置x轴刻度为整数
+        ax.set_xticklabels(x[accept_or_reject], rotation=30)
+        
+        # plt.savefig(f'reason_distribution_{type}.png')
+        # plt.close()                  
+                
+def draw_bar_chart_baseline(ax, baseline_or_ground, accept_or_reject):
+    # if baseline_or_ground == 'Baseline':
+    #     with open('../simulated_review/reason_result/reason_percentage.json', 'r') as f:
+    #         reason_percentage = json.load(f)
+    #         type_data = reason_percentage['BASELINE'][accept_or_reject]
+    # elif baseline_or_ground == 'Ground Truth':
+    with open('reason_percentage.json', 'r') as f:
+        reason_percentage = json.load(f)
+        type_data = reason_percentage[baseline_or_ground][accept_or_reject]
+    
+    
+    x = {
+        "accept": ['Novelty', 'Significance', 'Theoretical', 'Clarity', 'Future'],
+        "reject": ['Novelty', 'Theoretical', 'Validation', 'Practicality', 'Limitations', 'Presentation', 'Related Work']
+    }
+    x_range = range(1, len(x[accept_or_reject])+1)
+        
+    # 按照key排序
+    type_data = dict(sorted(type_data.items(), key=lambda x: int(x[0])))
+    
+    # dict中key应该是1-7，如果有的Key没有，就加上这个key，value设置为0
+    for i in x_range:
+        if str(i) not in type_data:
+            type_data[str(i)] = 0
+    
+    # 画图，将单一类型画到图上，选取颜色，设置透明度
+    width = 0.35  # 柱子的宽度
+    
+    # fig, ax = plt.subplots()
+    ax.bar(x_range, type_data.values(), width, label=accept_or_reject, color=COLORS[0], alpha=0.7)
+    
+    ax.legend()
+    ax.set_xlabel('Reason', fontsize=FONT_SIZE)
+    # ax.set_ylabel('Percentage', fontsize=FONT_SIZE)
+    ax.set_title(baseline_or_ground, fontsize=FONT_SIZE)
+    ax.set_xticks(x_range)  # 设置x轴刻度为整数
+    ax.set_xticklabels(x[accept_or_reject], rotation=30)
+    
+    # plt.savefig(f'{baseline_or_ground}_{accept_or_reject}_reason_distribution.pdf')
+    # plt.close()
+
+def draw_reason_distribution(accept_or_reject):
+    type2name = {'accept': 'Acceptance', 'reject': 'Rejection'}
+    
+    fig, axs = plt.subplots(1, 3, figsize=(15, 5))
+    fig.suptitle(f'Distribution of {type2name[accept_or_reject]} Reasons', fontsize=FONT_SIZE)
+    
+    # authoritarian_ACx1 inclusive_ACx1 conformist_ACx1
+    draw_bar_chart_baseline(axs[0], 'authoritarian_ACx1', accept_or_reject)
+    draw_bar_chart_baseline(axs[1], 'inclusive_ACx1', accept_or_reject)
+    draw_bar_chart_baseline(axs[2], 'conformist_ACx1', accept_or_reject)
+    
+    # draw_bar_chart_baseline(axs[0], 'Baseline', accept_or_reject)
+    # draw_bar_chart_baseline(axs[1], 'Ground Truth', accept_or_reject)
+    
+    # for i, (key, value) in enumerate(types.items()):
+    #     if i == 3:
+    #         break
+    #     draw_bar_chart(accept_or_reject, axs[i], key, value[0], value[1])
+
+    axs[0].set_ylabel('Percentage', fontsize=FONT_SIZE)
+    
+    plt.tight_layout()
+    plt.savefig(f'reason_distribution_AC_{accept_or_reject}.pdf')
+    plt.close()
+        
+
+if __name__ == "__main__":
+    # analysis_pipeline()
+    # convert_txt_to_json()
+    draw_reason_distribution('reject')
+    
+    
+# if __name__ == "__main__":
+#     # get current path
+#     # print(os.getcwd())
+#     print("Start analysis...")
+        
+#     json_files = []
+#     for root, dirs, files in os.walk(base_dir):
+#         for file in files:
+#             if file.endswith('.json'):
+#                 json_files.append(os.path.join(root, file))
+    
+#     # json_files = [f for f in json_files]
+#     # print(json_files)
+    
+#     # 将其平均分为6份，每份分配给一个进程
+#     n = len(json_files)
+#     n_per_process = n // 6
+#     processes = []
+#     for i in range(6):
+#         start = i * n_per_process
+#         end = (i + 1) * n_per_process
+#         if i == 5:
+#             end = n
+#         p = multiprocessing.Process(target=analyze_reason_in_batch, args=(json_files[start:end], ))
+#         processes.append(p)
+#         p.start()
+    

review_content_analysis/classification_prompt.txt

@@ -0,0 +1,58 @@
+You are outstanding data analysts. Now you need to analyze the reason of acceptance and rejection. Next is a review for a paper:
+{review}
+
+Here are some common reasons, please determine which of the following reasons appear in the review.
+
+Reasons for Acceptance
+1. Novelty and Innovation
+    - Introduces a new framework, method, or approach.
+    - Provides a unique perspective or solution to a problem.
+    - Advances the state-of-the-art in the field.
+2. Significance
+    - Addresses a relevant and important problem.
+    - Has potential practical applications or implications.
+    - Offers significant improvements over existing methods.
+3. Theoretical and Experimental Rigor
+    - Well-grounded in solid theoretical concepts.
+    - Provides thorough experimental validation.
+    - Includes comparisons with several baselines and ablations.
+4. Clarity and Motivation
+    - Clearly formulates the problem and solution.
+    - Motivates the approach with strong reasoning.
+    - Presents results that convincingly demonstrate effectiveness.
+5. Potential for Further Research
+    - Opens up new avenues for research.
+    - Can inspire future work in the field.
+
+Reasons for Rejection
+1. Lack of Novelty
+    - Does not offer a new contribution.
+    - Similar to existing work without significant improvements.
+    - Fails to differentiate from established methods.
+2. Insufficient Theoretical Foundation
+    - Lacks theoretical analysis or grounding.
+    - No proofs or discussions on convergence and stability.
+    - Unclear theoretical implications of the method.
+3. Inadequate Experimental Validation
+    - Limited or unconvincing experimental results.
+    - Lacks comparisons with strong baselines or state-of-the-art methods.
+    - Uses environments that do not capture real-world complexities.
+4. Scalability and Practicality Issues
+    - Does not address computational complexity or scalability.
+    - Unclear how the method performs with large or high-dimensional action spaces.
+    - Potential practical limitations not discussed.
+5. Insufficient Discussion of Limitations
+    - Does not explore potential drawbacks or failure modes.
+    - Lacks discussion on when the method may not perform well.
+    - No investigation of the impact of key parameters.
+6. Clarity and Presentation Issues
+    - Poorly articulated problem and solution.
+    - Dense or hard-to-follow sections.
+    - Missing or unclear figures and tables.
+7. Lack of Related Work Comparison
+    - Does not adequately compare with related work.
+    - Fails to position contributions within the broader context.
+    - Lacks comprehensive discussion on how it advances the field.
+
+Only output the final reason list, for example:
+"Accept: 1,3,5; Reject: 2,4,7"