| --- |
| library_name: transformers |
| tags: [] |
| --- |
| |
| # Model Card for Model ID |
|
|
| <!-- Provide a quick summary of what the model is/does. --> |
|
|
| CodeThinker is a fine-tuned LLM based on Qwen2.5-Instruct-32B, specializing in code generation. |
|
|
| # Inference |
| ``` |
| instruct_prompt = """Your role as a coding assistant is to provide thorough, well-reasoned, and precise solutions to coding questions by following a systematic long-reasoning process. Please think step-by-step and carefully before arriving at the final answer. Use the following step-by-step workflow: |
| |
| @@ Detailed Instructions: |
| |
| - Understanding the Problem: Begin by thoroughly reading and analyzing the task description, clarifying requirements and constraints. |
| - Planning: Devise a solution strategy by breaking down the task into smaller, manageable subproblems. Compare potential approaches and select the most efficient one based on constraints. |
| - Design: Outline the steps needed to implement the chosen approach. This could include pseudocode, notes on algorithms, or identifying edge cases. |
| - Implementation: Write clean, readable, and well-documented code. Use meaningful variable names and adhere to best practices. |
| - Testing: Test the code with a variety of cases, including edge cases, to ensure correctness and robustness. |
| - Optimization Cycle: |
| - Identify Failure: Encounter a realistic issue or inefficiency during testing. |
| - Analyze Root Cause: Understand why the failure occurred or why the solution is suboptimal. |
| - Refactor or Redesign: Make the necessary changes to address the issue or improve performance. |
| - Retest: Verify the improved solution. |
| - Share the polished solution under the header: ## Final Solution. Include the final implementation, neatly wrapped in code blocks with the header: ### Solution Code and followed by ```python[code]``` |
| |
| @@ Coding Question |
| {}""" |
| |
| from vllm import LLM, SamplingParams |
| from transformers import AutoTokenizer |
| |
| model = LLM( |
| model = "Chengran98/codethinker", |
| tensor_parallel_size=N, # N refers to the number of GPUs you wish to use |
| tokenizer = "Qwen/Qwen2.5-32B-Instruct", |
| dtype="auto", |
| enable_prefix_caching=True, |
| trust_remote_code=True, |
| ) |
| |
| sampling_params = SamplingParams( |
| max_tokens=32768, |
| temperature=0.7, |
| ) |
| |
| prompt = instruct_prompt+{user promt} |
| |
| response = model.generate(prompt, sampling_params=sampling_params) |
| |
| ``` |
|
|
| # Evaluation |
| The final solution provided in this version of CodeThinker is currently found under: |
| ``` |
| ## Final Solution -> ### Solution Code -> ```python[solution]``` |
| ``` |
| [TO DO]: An upcoming release will include improvements by wrapping the code solution within a specialized token `<solution>` during inference to enhance parsing capabilities. |