Salesforce describes agents for AI research and model improvement

Salesforce says the system takes a goal, a system to work on and a compute budget, then formulates hypotheses, implements methods and runs controlled experiments against baselines.[1] Its AutoR&D-Engineer agent searches existing code for algorithmic and systems improvements, while TrainForge explores training methods and evaluates candidates against held-out benchmarks the system does not train on.[1] AutoInfra provides the agents’ training backbone and runs experiments across shared GPU capacity.[1] Salesforce says the system retains codebase improvements only when evidence supports them and they are proven accurate and safe.[1]

Salesforce says SFR-AutoR&D autonomously improved SFR-Embedding V3, which it uses for retrieval in Data Cloud.[1] A subsequent run on EmbeddingGemma 2 produced SFR-Embedding v4, which Salesforce says improved both quality and latency.[1] The company reports that the research cycle took about 80 hours and that the model’s nDCG@10 score rose by 6.13 points across 10 public retrieval tasks in MTEB.[1] Salesforce also reports a separate session focused on speed and cost: it lasted roughly nine hours and increased throughput by 36% while reducing latency by 17%, with almost no loss in quality.[1]

Embedding models convert text into numerical representations that help systems find relevant records, documents or case histories among millions of items, according to Salesforce.[1] The company says that when an Agentforce agent answers a question grounded in a customer’s data, it reaches through Data Cloud, where an embedding model determines what comes back.[1] That makes retrieval quality and response speed relevant measures for these workflows, though the reported results do not establish how every deployment will perform.[1]

Salesforce says researchers set the system’s goal, budget, evaluation and data constraints, and that the agents explore within those limits.[1] The company says models trained for product use draw only on datasets that have passed ethical and legal review.[1] Salesforce also cautions that a model performing well in evaluation still must clear production requirements, and says that work is underway.[1] The research site reports additional results, including up to 1.84 times faster nullable-dtype reductions in pandas and 17.5% more queries per second for USearch in a single-core test using GCC 11 and one million vectors.[2] An interactive platform is coming soon, according to the research site.[3]