We examine differentially non-public (DP) optimization algorithms for stochastic and empirical targets that are neither easy nor convex, and suggest strategies that return a Goldstein-stationary level with pattern complexity bounds that enhance on current works.
We begin by offering a single-pass -DP algorithm that returns an -stationary level so long as the dataset is of measurement , which is occasions smaller than the algorithm of Zhang et al. [2024] for this process, the place is the dimension.
We then present a multi-pass polynomial time algorithm which additional improves the pattern complexity to , by designing a pattern environment friendly ERM algorithm, and proving that Goldstein-stationary factors generalize from the empirical loss to the inhabitants loss.
†Work partially finished throughout Apple internship
