🚀 Exciting AI Updates: Adobe just added generative AI capabilities to Photoshop | Meta, CMU, USC, and Tel Aviv University Propose LIMA | Purely Synthetic LLM Feedback Improves LLMs....

CoT Collection: An instruction dataset including 52 times more CoT rationales and 177 times more tasks compared to previously available CoT datasets. Recent works showed that CoT Fine-tuning enables smaller LMs to solve new novel tasks more effectively. But, up to date, there exists only 9 CoT datasets available, which FLAN-T5 used for training, namely AQuA, Creak, ECQA, eSNLI, GSM8K, QASC, QED, SenseMaking and StrategyQA. To resolve this issue, researchers from KAIST augment CoT rationales using LLMs. Using FLAN Collection as a source, the research team use ICL and make LLMs generate high-quality CoT rationales across 1,060 NLP tasks.

Adobe just added generative AI capabilities to Photoshop: The new AI tools from Adobe will work similarly to other AI image creation technology, such as Midjourney and DALL-E, which allow people to produce pictures from simple text prompts. The new Firefly-powered Generative Fill is the world’s first co-pilot in creative and design workflows, giving users a magical new way to work by easily adding, extending or removing content from images non-destructively in seconds using simple text prompts. This beta release of Photoshop is Adobe’s first Creative Cloud application to deeply integrate Firefly with an exciting roadmap ahead that will transform workflows across Creative Cloud, Document Cloud, Experience Cloud and Adobe Express.

Meta, CMU, USC, and Tel Aviv University Propose LIMA - a new 65B parameter LLaMa model fine-tuned on 1000 carefully curated prompts and responses. It doesn't use RLHF. It generalizes well to unseen tasks not in the training data. LIMA responses are equivalent or preferred to GPT-4 in 43% of cases, and even higher compared to Bard and davinci003. It's remarkable you can get high-quality outputs with such a simple approach and limited instruction tuning. While LIMA shows strong performance across a wide range of prompts and tasks, scaling up examples is challenging

Purely Synthetic LLM Feedback Improves LLMs: The AI research put forth presents a cutting-edge blueprint for learning alignment with little to no human involvement and without the need for pre-aligned large language models (LLMs). The process begins with reward modeling (RM) using artificial feedback, which is achieved by comparing responses from basic LLMs of different sizes and prompts. Subsequently, this RM is used to generate high-quality demonstrations for training a supervised policy and for additional optimization of the model through reinforcement learning. The product of this process is the Aligned Language Model with Synthetic Training dataset (ALMoST), which outclasses public models such as Alpaca, Dolly, and OpenAssistant, which are trained using the outputs of InstructGPT or human-annotated instructions. Remarkably, the 7B-sized model outperforms 12-13B models in A/B tests where GPT-4 serves as the adjudicator, boasting an average win rate of about 75%.

Meet this set of highly customizable and open-source RL environments, ChemGymRL, based on the standard Open AI Gym template. ChemGymRL facilitates a network of interlinked virtual chemical workstations where Reinforcement Learning (RL) agents can function and learn. The paper presents and elaborates on each workstation, utilizing familiar chemical reactions as demonstration examples and training an array of conventional RL algorithms within each workstation. It concludes with an analysis and comparison of the effectiveness of various standard RL techniques. Additionally, it offers a set of recommendations for future exploration, framing a blueprint for the ongoing evolution and application of ChemGymRL.