Skip to content

Code and datasets for paper "GeoGalactica: A Scientific Large Language Model in Geoscience"

License

Notifications You must be signed in to change notification settings

geobrain-ai/geogalactica

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Ge🌏Galactica

A Scientific Large Language Model in Geoscience

  • Technical report is HERE!
  • The data pre-processing toolkits are open sourced on sciparser!

Statement

To clarify with potential confusions, we hereby state that the model with the manuscript "GeoGalactica: A Scientific Large Language Model in Geoscience" is not associated with the DDE Program, nor supported by DDE related fundings. We feel sorry for the unintentional misunderstandings and inconvenience, and we commit to prevent future misunderstandings.

Introduction

GeoGalactica is from further pre-training of Galactica -- a top-performing LLM trained with a large number of scientific documents. In this work, we take the initial step to leverage LLM for science, through a rather straightforward approach. We try to specialize an open-sourced LLM into geoscience, by further pre-training the model with a vast amount of texts in geoscience, as well as supervised fine-tuning (SFT) the resulting model with our custom collected instruction tuning dataset. These efforts result in a model GeoGalactica consisting of 30 billion parameters. To our best knowledge, it is the largest language model for the geoscience domain.

Resources

Quick Start

A simple script is provided (tools/prediction/demo.py) for the model to predict the output text for a single input. The memory exceeds 140GB. The folder example_data shares data file format during the training.

Contributors

This project was founded by Acemap at Shanghai Jiao Tong University, leading by Zhouhan Lin and a group of students including Cheng Deng* (student leader), Le Zhou, Tianhang Zhang, Yi Xu, Yutong Xu, Beiya Dai, Qiyuan Chen, Yuanyuan Shi and Zhongmou He supervised by Zhouhan Lin, Junxian He, Xinbing Wang, and Chenghu Zhou.

Acknowledgements

GeoGalactica has referred to the following open-source projects. We want to express our gratitude and respect to the researchers of the projects.

We would also like to express our appreciation for the effort of data processing and annotation from the students in CAS.

License

GeoGalactica is a research preview intended for non-commercial use only, subject to the model License of Galactica and the Terms of Use of the data generated by OpenAI. Please contact us if you find any potential violations. The code is released under the Apache License 2.0. The data GeoSignal and GeoBench is open-sourced by K2.

About

Code and datasets for paper "GeoGalactica: A Scientific Large Language Model in Geoscience"

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages