|
|
---
|
|
|
license: mit
|
|
|
---
|
|
|
|
|
|
# PyAutoCode: GPT-2 based Python auto-code.
|
|
|
|
|
|
PyAutoCode is a cut-down python autosuggestion built on **GPT-2** *(motivation: GPyT)* model. This baby model *(trained only up to 3 epochs)* is not **"fine-tuned"** yet therefore, I highly recommend not to use it in a production environment or incorporate PyAutoCode in any of your projects. It has been trained on **112GB** of Python data sourced from the best crowdsource platform ever -- **GitHub**.
|
|
|
|
|
|
*NOTE: Increased training and fine tuning would be highly appreciated and I firmly believe that it would improve the ability of PyAutoCode significantly.*
|
|
|
|
|
|
## Some Model Features
|
|
|
|
|
|
- Built on *GPT-2*
|
|
|
- Tokenized with *ByteLevelBPETokenizer*
|
|
|
- Data Sourced from *GitHub (almost 5 consecutive days of latest Python repositories)*
|
|
|
- Makes use of *GPTLMHeadModel* and *DataCollatorForLanguageModelling* for training
|
|
|
- Newline characters are custom coded as `<N>`
|
|
|
|
|
|
## Get a Glimpse of the Model
|
|
|
|
|
|
You can make use of the **Inference API** of huggingface *(present on the right sidebar)* to load the model and check the result. Just enter any code snippet as input. Something like:
|
|
|
|
|
|
```sh
|
|
|
for i in range(
|
|
|
```
|
|
|
|
|
|
## Usage
|
|
|
|
|
|
You can use my model too!. Here's a quick tour of how you can achieve this:
|
|
|
|
|
|
Install transformers
|
|
|
```sh
|
|
|
$ pip install transformers
|
|
|
```
|
|
|
|
|
|
Call the API and get it to work!
|
|
|
```python
|
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained("P0intMaN/PyAutoCode")
|
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("P0intMaN/PyAutoCode")
|
|
|
|
|
|
# input: single line or multi-line. Highly recommended to use doc-strings.
|
|
|
inp = """import pandas"""
|
|
|
|
|
|
format_inp = inp.replace('\n', "<N>")
|
|
|
tokenize_inp = tokenizer.encode(format_inp, return_tensors='pt')
|
|
|
result = model.generate(tokenize_inp)
|
|
|
|
|
|
decode_result = tokenizer.decode(result[0])
|
|
|
format_result = decode_result.replace('<N>', "\n")
|
|
|
|
|
|
# printing the result
|
|
|
print(format_result)
|
|
|
```
|
|
|
|
|
|
Upon successful execution, the above should probably produce *(your results may vary when this model is fine-tuned)*
|
|
|
```sh
|
|
|
import pandas as pd
|
|
|
import numpy as np
|
|
|
import matplotlib.pyplot as plt
|
|
|
|
|
|
```
|
|
|
## Credits
|
|
|
##### *Developed as a part of a university project by [Pratheek U](https://www.github.com/P0intMaN) and [Sourav Singh](https://github.com/Sourav11902312lpu)* |
