UC-01
INGEST OPPORTUNITIES
Collect and normalize software offers into a consistent local dataset.
SYSTEM 02
KULAI STUDIO
LOCAL AI / HUMAN-IN-THE-LOOP / PERSONALIZED ML
Local-first software opportunity intelligence and project automation system connecting offer ingestion, structured LLM analysis, human feedback, reusable capability matching and controlled personalized ML experiments.
SYSTEM IDENTITY
SYSTEM IDKULAI-STUDIO-02
STATUSACTIVE SYSTEM / ML R&D
TYPELOCAL AI / ML / AUTOMATION
Opportunity intelligence system connecting structured AI analysis, human feedback, personalized learning and reusable software delivery.
01 / OVERVIEW
FROM OFFER INTAKE TO A LEARNING SYSTEM
KulAI Studio started as a way to organize reusable software delivery. It has grown into a local-first opportunity intelligence system connecting source acquisition, normalization, local LLM analysis, capability matching, human review, ranking, ML research and delivery tooling.
The system separates two questions: what does this project actually require, and is this project a good opportunity for this specific developer and workflow? The local LLM primarily addresses the first question. Personalized ML research addresses the second.
Every manually reviewed offer can become additional supervised evidence for future evaluation. The intended loop is controlled: collect new human labels, retrain a fresh candidate on the full dataset, compare it with the baseline and current model, then promote only after evidence.
02 / PROBLEM
WHAT THE SYSTEM IS SOLVING
UC-02
STRUCTURE UNSTRUCTURED REQUIREMENTS
Use a local language model to turn natural-language briefs into validated structured requirements.
UC-03
MATCH REUSABLE CAPABILITIES
Compare requirements against reusable KulAI Modules and identify existing, extendable and project-specific work.
UC-04
CAPTURE HUMAN JUDGMENT
Record explicit human decisions, ratings, tags and rationale rather than treating model output as ground truth.
UC-05
LEARN PERSONAL PREFERENCES
Use supervised ML experiments to learn which automatically observable offer signals correlate with real human decisions.
UC-06
MOVE INTO DELIVERY
Connect selected opportunities to reusable project-generation and delivery workflows.
03 / SYSTEM PIPELINE
OPPORTUNITY INTELLIGENCE PIPELINE
DATA FLOW
- OFFER SOURCE
- NORMALIZATION
- POSTGRESQL
- LOCAL QWEN ANALYSIS
- STRUCTURED REQUIREMENTS
DECISION FLOW
- STRUCTURED ANALYSIS
- HUMAN REVIEW
- RANKING / ML
- PROJECT SELECTION
- DELIVERY
The data flow has a clear responsibility boundary: the local LLM converts unstructured offer text into a versioned structured semantic representation. Qwen is not presented as the personalized decision model.
Personalized ML consumes automatically available offer features and predicts human preference. It answers a different question from semantic parsing and is evaluated separately.
04 / HUMAN FEEDBACK
HUMAN FEEDBACK AS SUPERVISION
Model output is not treated as ground truth. The review workspace stores APPLY / MAYBE / REJECT decisions, six independent 1-10 dimensions, tags, free-form rationale and notes, append-only review history, and independent offer workflow state.
HF-01
DECISION LABEL
APPLY / MAYBE / REJECT is the primary supervised target.
HF-02
MULTI-DIMENSIONAL RATINGS
Six separate dimensions preserve more information than one aggregate score: interest, technical fit, profitability, scope quality, KulAI value and win chance.
HF-03
RATIONALE
Tags and free-form notes preserve why two apparently similar opportunities may lead to different decisions.
HF-04
HISTORY
Review revisions remain historical rather than being silently overwritten, alongside independent workflow state.
Human ratings and notes are not used directly as E1 inference features because they do not exist before a new offer is reviewed. They are supervision and evaluation evidence, not leaked inputs.
05 / ML R&D
PERSONALIZED OFFER ML R&D
Can automatically available offer information predict real APPLY / MAYBE / REJECT decisions? The research deliberately compares increasingly richer representations instead of assuming that a more complex model is automatically better.
E0
DETERMINISTIC BASELINE · IMPLEMENTED
Existing hand-designed Offer Ranking V1. It is the baseline that learned models must beat.
E1
STRUCTURED ML · COMPLETED
A 31-feature structured Logistic Regression baseline evaluated with deterministic out-of-fold predictions.
E2
LEXICAL TEXT · COMPLETED
E1 plus leakage-safe TF-IDF from canonical title and description, evaluated on the same folds.
BENCHMARK
FORMAL PAIRED BENCHMARK · COMPLETED
E1 and E2 compared on 123 shared samples. E1 remains rank 1 on macro-F1; E2 is not presented as the overall winner.
E3
SEMANTIC TEXT · PLANNED
E1 plus pretrained semantic text embeddings, measuring whether semantic similarity adds information beyond lexical features.
E4
RATIONALE-AWARE · PLANNED
Historical notes provide privileged supervision. Notes are not directly required at inference time; future models may learn rationale dimensions from the offer itself.
E5
RATING-AWARE · PLANNED
Predict the six human rating dimensions from offer data and test whether out-of-fold predicted ratings improve final decision ranking.
06 / EXPERIMENT DESIGN
CONTROLLED EVALUATION
RND-01
FROZEN DATASET
Each historical offer is represented once using its latest human review and latest valid structured analysis.
RND-02
LEAKAGE CONTROL
Human decision, ratings, notes and workflow state are not E1 input features because they do not exist before a new offer is reviewed.
RND-03
SHARED CV SPLITS
Experiments use the same deterministic stratified folds for fair comparison.
RND-04
OUT-OF-FOLD PREDICTIONS
Training rows are evaluated only by models that did not train on those rows.
RND-05
RANKING METRICS
Evaluation will cover classification and queue quality, including Precision@K, NDCG@K, Macro F1 and Balanced Accuracy. Values are not published before measurement.
RND-06
SHADOW EVALUATION
Planned prospective test: a new offer is predicted before review, the prediction stays hidden, the human labels independently, and only then is the prediction evaluated.
07 / INTERFACE
KULAI STUDIO INTERFACE
The current UI covers the dashboard, project creation, offer acquisition and synchronization, offer review, human decisions and independent workflow tracking. It does not present an ML prediction dashboard yet.
01 / 03
08 / ENGINEERING
ENGINEERING DECISIONS
ENG-01
LOCAL-FIRST AI
Offer interpretation runs against a local language model instead of requiring a hosted inference dependency.
ENG-02
STRUCTURED LLM BOUNDARY
LLM output is validated into versioned structured contracts rather than being treated as arbitrary prose.
ENG-03
HUMAN GROUND TRUTH
Human decisions remain authoritative and separate from model predictions.
ENG-04
DETERMINISTIC BASELINE
A hand-designed ranking baseline remains available so ML improvements can be measured rather than assumed.
ENG-05
REPRODUCIBLE R&D
Frozen dataset fingerprints, deterministic splits and OOF evaluation make experiments comparable.
ENG-06
REUSABLE DELIVERY
KulAI Modules and project-generation paths connect analysis with actual software delivery.
09 / TECH STACK
TECHNICAL LAYER
STACK-01
PYTHON / FASTAPI
Local control plane, application logic and automation.
STACK-02
POSTGRESQL / SQLALCHEMY
Normalized offer, analysis, review and workflow persistence.
STACK-03
LOCAL QWEN 27B
Semantic parser for structured offer analysis.
STACK-04
PYDANTIC
Strict structured contracts around model output and application boundaries.
STACK-05
CLASSICAL MACHINE LEARNING
Supervised personalization research starting with an interpretable Logistic Regression baseline.
STACK-06
KULAI MODULES
Reusable capability catalog and software delivery foundation.
10 / STATUS
CURRENT SYSTEM / RESEARCH ROADMAP
WORKING SYSTEM OFFER INGESTION / STRUCTURED LOCAL LLM ANALYSIS / CAPABILITY MATCHING / HUMAN REVIEW / WORKFLOW TRACKING / DETERMINISTIC RANKING / PROJECT DELIVERY TOOLING
COMPLETED R&D E1 STRUCTURED BASELINE / E2 STRUCTURED + TF-IDF / FORMAL PAIRED BENCHMARK
NEXT DATA STRATEGY MORE INDEPENDENT REVIEWS / DECISION-BOUNDARY EXAMPLES / HISTORICAL FEATURE SNAPSHOTS