This paper introduces POLCA, a scalable framework designed to automate the optimization of complex systems like LLM prompts and multi-turn agents. The authors formalize this challenge as stochastic generative optimization, where an LLM acts as the optimizer but must contend with noisy feedback, random system behaviors, and an ever-expanding solution space. To ensure efficiency, POLCA utilizes a priority queue to balance exploration and exploitation alongside an $\epsilon$-Net mechanism that prunes semantically redundant candidates. A specialized LLM Summarizer also performs meta-learning by compressing historical successes and failures into a global context for future iterations. Theoretical analysis proves the framework converges to near-optimal solutions despite stochasticity, and experimental results across benchmarks like $\tau$-bench and VeriBench show it consistently outperforms existing state-of-the-art algorithms. Ultimately, the research highlights how embedding-based memory and systematic filtering are essential for making generative optimization robust and computationally feasible.