AI 뉴스

Simple considerations for simple people building fancy neural networks

Retrieval Augmented Generation with Huggingface Transformers and Ray

Hugging Face on PyTorch / XLA TPUs

Understanding the capabilities, limitations, and societal impact of large language models

Faster TensorFlow models in Hugging Face Transformers

Scaling Kubernetes to 7,500 nodes

We’ve scaled Kubernetes clusters to 7,500 nodes, producing a scalable infrastructure for large models like GPT-3, CLIP, and DALL·E, but also for rapid small-scale iterative research such as Scaling Laws for Neural Language Models.

Fit More and Train Faster With ZeRO via DeepSpeed and FairScale

How we sped up transformer inference 100x for 🤗 API customers

DALL·E: Creating images from text

We’ve trained a neural network called DALL·E that creates images from text captions for a wide range of concepts expressible in natural language.

CLIP: Connecting text and images

We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized, similar to the “zero-shot” capabilities of G…

Organizational update from OpenAI

It’s been a year of dramatic change and growth at OpenAI.

Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models

Porting fairseq wmt19 translation system to transformers

Hyperparameter Search with Transformers and Ray Tune

Transformer-based Encoder-Decoder Models

OpenAI licenses GPT-3 technology to Microsoft

OpenAI has agreed to license GPT-3 to Microsoft for their own products and services.

Block Sparse Matrices for Smaller and Faster Language Models

Generative language modeling for automated theorem proving

Learning to summarize with human feedback

We’ve applied reinforcement learning from human feedback to train language models that are better at summarization.

OpenAI Scholars 2020: Final projects

Our third class of OpenAI Scholars presented their final projects at virtual Demo Day, showcasing their research results from over the past five months.