Creating a fine-tuned model

You've learned what fine-tuning is and when it makes sense for your application. Now, let's walk through the practical steps of actually fine-tuning a model, from preparing your data to deploying it in production. This pipeline applies whether you're fine-tuning GPT models through OpenAI's API, running Mistral on AWS Bedrock, or training an open-source Llama model on your own hardware.

Prerequisites for fine-tuning

Before you start a fine-tuning job, you need the right foundation in place. The quality of your results depends heavily on what you bring to the table before any training begins. First and foremost, you need a training dataset that accurately represents the task you want the model to learn. This dataset should contain enough examples, though the exact number varies by task complexity and model size. Here, quality matters more than quantity. Each example should demonstrate the exact behavior you want the model to replicate.

Next, you need to choose your base model. Different foundation models have different strengths, licensing terms, and cost structures. OpenAI's GPT models excel at general reasoning and instruction-following, but they come with API costs and require you to send data to OpenAI's servers. Cohere's models offer strong multilingual capabilities and enterprise features. Open-source models like Llama, Mistral, or Qwen give you full control and can run in your own infrastructure, but they require more technical setup. Consider factors like model size, licensing restrictions (especially for commercial use), and whether you need on-premises deployment for data privacy.

Your infrastructure and compute resources must match your chosen approach. If you're using managed fine-tuning services from OpenAI, Cohere, or cloud providers like AWS Bedrock or Google Cloud Vertex AI, you only need to prepare your data and configure the job through their APIs. For open-source models, you need GPU access. Full fine-tuning of large models demands high-end GPUs with substantial VRAM—think NVIDIA A100s or H100s. Parameter-efficient methods, such as LoRA and QLoRA, make fine-tuning possible on more modest hardware.

Finally, prepare your evaluation strategy before training starts. You need a held-out validation set—typically 10-20% of your data—that the model never sees during training. Define clear success metrics for your use case. For classification tasks, accuracy, precision, recall, and F1 score indicate how well the model distinguishes categories. For generation tasks, metrics such as BLEU, ROUGE, and perplexity provide quantitative signals, but you also need human and model evaluation to assess coherence, relevance, and style. To compare different training runs, set up logging and tracking tools like Weights & Biases early.

Preparing your training data

No amount of hyperparameter tuning can compensate for poorly formatted, noisy, or misaligned training examples. Start by structuring your data in the format your chosen platform expects. OpenAI requires JSONL (JSON Lines) files where each line is a JSON object containing a messages array with role-based conversation turns. If working with vision models, image inputs are used. Cohere uses a similar structure but with different field names. Hugging Face datasets typically use conversational formats or simple text pairs, depending on the model architecture. Here’s an example for OpenAI models:

{"messages": [{"role": "user", "content": "Classify the sentiment of this review: The product arrived damaged and customer service was unhelpful."}, {"role": "assistant", "content": "Sentiment: Negative\\nConfidence: High\\nKey issues: Product quality, Customer service"}]}
{"messages": [{"role": "user", "content": "Classify the sentiment of this review: Great quality and fast shipping! Exactly what I needed."}, {"role": "assistant", "content": "Sentiment: Positive\\nConfidence: High\\nKey aspects: Product quality, Shipping speed"}]}
{"messages": [{"role": "user", "content": "Classify the sentiment of this review: It's okay, nothing special but does the job."}, {"role": "assistant", "content": "Sentiment: Neutral\\nConfidence: Medium\\nKey aspects: Functional, Average quality"}]}

When working with vision models, some models, such as Amazon Rekognition, require image datasets with bounding boxes for object detection. For OpenAI models, image inputs are used:

{"messages": [{"role": "system", "content": "You are an assistant that identifies plant diseases from leaf images."}, {"role": "user", "content": "What disease is affecting this plant?"}, {"role": "user", "content": [{"type": "image_url", "image_url": {"url": "https://example.com/images/tomato-leaf-blight.jpg"}}]}, {"role": "assistant", "content": "Early Blight (Alternaria solani). Recommended treatment: Remove affected leaves, apply copper-based fungicide, and ensure proper spacing for air circulation."}]}

Once your data is formatted correctly, validate it thoroughly. Check for duplicates—repeated examples waste training time and can cause the model to overfit. Look for labeling errors or inconsistencies where similar inputs have different outputs without a good reason. OpenAI provides a validation tool that checks your JSONL file for format errors and estimates token usage before you upload it. Catching format issues early saves time and API costs.

Data balancing matters for classification and structured output tasks. If your training set has 90% examples of one category and only 10% of another, the model will become biased toward the majority class. Either collect more examples of underrepresented classes or use techniques such as oversampling or undersampling to create a more balanced distribution. Also, create an 80-20 or 90-10 split of your data into training and validation sets. Keep the validation set separate and never let the model see it during training.

Start a fine-tuning job

With your data prepared and validated, you can kick off the actual training process. Each platform has its own API or interface for submitting fine-tuning jobs, but the core concept remains the same. You upload your training data, specify the base model, configure hyperparameters, and let the platform handle the rest.

Hyperparameters control how the model learns during fine-tuning, and getting them right makes the difference between effective learning and poor results. Key parameters include learning rate (controls weight updates per step), batch size (examples processed together), number of epochs (passes over the dataset), and warmup steps (gradual learning rate increase at start). For LoRA methods, you'll also set rank (r, controlling parameter count—start with 16) and alpha (scaling factor—typically 2*r). Start with sensible defaults, train a baseline model, then adjust one or two parameters at a time based on your metrics.

Here’s how you can start a training job for different providers:

from openai import OpenAI

client = OpenAI()

# Upload training file
training_file = client.files.create(
    file=open("training_data.jsonl", "rb"),
    purpose="fine-tune"
)

# Create fine-tuning job
fine_tune_job = client.fine_tuning.jobs.create(
    training_file=training_file.id,
    model="gpt-4.1-nano-2025-04-14",
    hyperparameters={
        "n_epochs": 3,
        "learning_rate_multiplier": 0.5
    }
)

print(f"Fine-tune job ID: {fine_tune_job.id}")
print(f"Status: {fine_tune_job.status}")

import cohere

co = cohere.Client(api_key="your-cohere-api-key")

# Upload your training dataset
dataset = co.datasets.create(
    name="customer-support-classifier",
    data=open("training_data.jsonl", "rb"),
    type="chat-finetune"
)

# Create a fine-tuning job
finetune = co.finetuning.create_finetuned_model(
    request={
        "name": "support-classifier-v1",
        "settings": {
            "base_model": "command",
            "dataset_id": dataset.id,
            "hyperparameters": {
                "train_epochs": 3,
                "learning_rate": 0.00001,
                "train_batch_size": 16
            }
        }
    }
)

print(f"Fine-tune job ID: {finetune.id}")
print(f"Status: {finetune.status}")

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from peft import LoraConfig, get_peft_model
from datasets import load_dataset

# Load base model and tokenizer
model_name = "mistralai/Mistral-7B-v0.1"
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", load_in_4bit=True)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Configure LoRA
lora_config = LoraConfig(
    r=16,  # Low-rank dimension
    lora_alpha=32,  # Scaling factor
    target_modules=["q_proj", "v_proj"],  # Which layers to adapt
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()  # Shows only ~0.5% of params are trainable

# Load and prepare dataset
dataset = load_dataset("json", data_files={"train": "train.jsonl", "validation": "val.jsonl"})

# Define training arguments
training_args = TrainingArguments(
    output_dir="./mistral-support-finetuned",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    fp16=True,  # Mixed precision training
    logging_steps=10,
    save_steps=100,
    evaluation_strategy="steps",
    eval_steps=100,
    report_to="wandb"  # Log metrics to Weights & Biases
)

# Initialize trainer and start fine-tuning
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"]
)

trainer.train()
trainer.save_model("./mistral-support-final")

Tracking and evaluating your fine-tuned model

Once training starts, you need to monitor progress and evaluate whether the model is actually improving. Different platforms offer different monitoring and evaluation tools, but the core principles remain the same across all providers. For proprietary APIs such as OpenAI and Cohere, evaluation happens through their managed dashboards and APIs. OpenAI provides training metrics directly in their fine-tuning dashboard, where you can view loss curves, step counts, and estimated completion times. You can also retrieve metrics programmatically to track validation loss and other signals.

For Cohere, the platform automatically calculates metrics like accuracy and loss during training. You can query these through the SDK or view them in the web dashboard. AWS Bedrock provides CloudWatch integration for monitoring fine-tuning jobs, logging metrics such as training loss, validation loss, and resource utilization. These managed platforms handle most of the evaluation infrastructure for you, but they often limit the custom metrics you can track or the granularity of insights you can access.

Open-source fine-tuning gives you complete control over evaluation. Tools like Weights & Biases and MLflow let you track every aspect of training in real time. With W&B, you can log training loss, validation loss, learning rate schedule, and training speed, which you can see in the dashboard and watch the curves update as training progresses. If something looks wrong—loss spiking, validation metrics not improving—you can stop the run early instead of waiting hours for a failed experiment to finish.

Once your model is deployed, evaluation doesn't stop—it actually becomes more critical. Production evaluation monitors how the model performs on real user queries, not just curated test sets. Set up logging to capture model inputs, outputs, and any user feedback signals like thumbs up/down ratings, edit rates, or task completion metrics. For OpenAI and Cohere deployments, you can track API usage metrics through their dashboards, but you'll need your own logging layer to capture application-specific feedback. Tools like Langfuse and LangSmith can help here.

Deploying your fine-tuned model

Finally, your fine-tuned model must handle real traffic, scale to meet demand, and integrate with your application. The deployment process varies significantly between proprietary APIs and self-hosted open-source models. For proprietary models like OpenAI's GPT or Cohere's Command, deployment is straightforward once training completes. The fine-tuned model becomes available through the same API you already use, but with a new model identifier. OpenAI provides a model ID like ft:gpt-4.1-2025-04-14:your-org:abc123 that you reference in your API calls.

from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()

client = OpenAI()

response = client.responses.create(
    model="ft:gpt-4.1-nano-2025-04-14:personal::D47uOdtj",
    instructions="You're a customer support assistant at Hyperskill.",
    input="How do I track my learning progress?"
)

print(response.output_text)

"""
Hyperskill provides several ways to track your progress:

1. **Dashboard**: Your personal dashboard displays completed projects, solved problems, and finished topics. It gives you an overview of what you've accomplished and what remains.
2. **Statistics**: The platform shows statistics like daily streaks, total solved problems, and time spent learning. These metrics motivate consistent practice.
3. **Achievements**: As you complete significant milestones, you earn badges and achievements that mark your progress.

...
"""

The API automatically routes requests to your fine-tuned model, which you pay for per token, just like the base model. This deployment model works well, especially if you value simplicity and managed infrastructure. The main tradeoff is vendor lock-in—you depend entirely on the provider's availability, pricing, and terms of service.

For open-source models, you have full control but more responsibility. After fine-tuning a model like Mistral or Llama, you need to deploy it on the infrastructure you manage. Cloud providers like AWS Bedrock provide a middle ground, letting you deploy fine-tuned models through managed infrastructure with autoscaling and secure multi-tenancy. For maximum control, you can self-host using tools like vLLM. Self-hosting eliminates per-token API costs and gives you complete control over versioning, monitoring, and data privacy. You can deploy on Kubernetes for autoscaling and add caching layers for repeated queries.

Fine-tuning risks

Fine-tuning isn't without risks. Understanding these pitfalls before you commit time and resources helps you avoid costly mistakes. One of the most common problems is overfitting, where the model learns your training data so well that it loses the ability to generalize. The model essentially memorizes specific examples instead of learning underlying patterns. You can mitigate overfitting by:

• Using enough training data.

• Keeping validation and training loss aligned.

• Enabling weight decay.

• Stopping training early when validation metrics stop improving.

• Testing on diverse examples that differ from your training set.

Data quality issues undermine everything. If your training data contains errors, biases, inconsistent labels, or examples that don't match your actual use case, the model learns those flaws and amplifies them. Always audit your data before fine-tuning, and have multiple reviewers check for labeling consistency. Remove outliers and edge cases that don't represent real usage. A model trained on leaked customer data or biased annotations creates legal and ethical risks that far outweigh any performance gains.

Catastrophic forgetting happens when fine-tuning erases useful knowledge the base model originally had. If you train too aggressively on a narrow task, the model might lose its ability to handle related but slightly different queries. Mitigate this by using moderate learning rates, training for fewer epochs, using PEFT methods like LoRA that modify fewer parameters, and testing the fine-tuned model on diverse inputs beyond your specific task. If the model's general capabilities degrade unacceptably, you may need to mix in some general instruction-following examples alongside your task-specific data.

Cost and resource management can spiral quickly, especially with proprietary APIs or full fine-tuning of large models. OpenAI's fine-tuning costs include both training and inference charges. Cloud GPU rentals for self-hosted training can also exceed budgets if you don't monitor usage. Set spending limits, estimate costs before starting, use smaller models or PEFT methods when possible, and shut down resources immediately after fine-tuning jobs are complete.

Conclusion

You've walked through the complete fine-tuning pipeline, from preparing training data and starting jobs across different providers to tracking metrics, deploying models, and navigating risks. We covered critical prerequisites like dataset quality and infrastructure, formatting and validating data, launching fine-tuning jobs on both proprietary and open-source platforms, and evaluation strategies. You also learned about deployment options ranging from managed APIs to self-hosted inference servers and common pitfalls like overfitting and catastrophic forgetting. With this foundation, you're ready to adapt foundation models to your specific needs.