Collect parallel data
Assemble sentence-aligned bilingual corpora and split into train/validation/test sets, keeping domain and register consistent.
Translate short sentences. Assess translation quality.
Neural machine translation (NMT) learns to map sentences in one language to another using parallel corpora. Important practical issues include segmentation (subword tokenisation), domain mismatch, out-of-vocabulary words, and evaluation that mixes automatic scores (BLEU) with targeted human checks for adequacy and fluency.
Learning concept · Not a validated service
Neural machine translation (NMT) learns to map sentences in one language to another using parallel corpora. Important practical issues include segmentation (subword tokenisation), domain mismatch, out-of-vocabulary words, and evaluation that mixes automatic scores (BLEU) with targeted human checks for adequacy and fluency.
A proposed application: provide draft translations for localisation teams, with post-editing by human linguists. This extension would require domain-specific parallel data and human review and is not delivered by the supplied starter code.
Assemble sentence-aligned bilingual corpora and split into train/validation/test sets, keeping domain and register consistent.
Apply subword tokenisation (BPE/WordPiece) to control vocabulary size and handle rare words before creating datasets.
Fit a small transformer or RNN encoder–decoder with attention, monitor validation loss and sample translations during training.
Report BLEU and TER, then perform targeted human checks for adequacy, fluency, and common error types like dropped information.
The supplied starter creates a synthetic numeric classification dataset with sklearn.make_classification, fits a scikit-learn Pipeline that scales features and trains a RandomForestClassifier, then prints a classification report. It contains no text data, no tokenisation, and no sequence-to-sequence model or inference for translation.
model = Pipeline(
steps=[
("scale", StandardScaler()),
("classifier", RandomForestClassifier(n_estimators=250, random_state=RANDOM_STATE)),
]
)
model.fit(X_train, y_train)
predictions = model.predict(X_test)The starter uses numeric synthetic features and a RandomForest classifier; it contains no parallel text data, no tokenisation or subword modelling, and no seq2seq/transformer components required for neural machine translation.
Proposed extension: prepare a small domain-specific parallel corpus, train a compact transformer (or fine-tune a pre-trained encoder–decoder), expose an inference API for draft translations and collect post-edits to iterate the model.
This is a potential application pathway, not a delivered product. The supplied starter does not implement this service or validate it with real users.
Networkers Home runs this topic inside its AI Engineer Course, with placement support and hands-on labs.