Our recent work describes cheap methods to train deep networks with
explicit constraints:
we call it Deep-Conditional Gradient (Deep-CG).
Our paper provides compelling evidence supporting three distinct but
closely related threads:
global constraints are highly relevant;
the lack of support for global constraints in existing libraries may be
because of the complex interplay between constraints and SGD which can be
effectively side-stepped using CG; and
li>
constraints can be easily incorporated in existing implementations.
Intuitively, a coreset is a subset of the original dataset
which behaves "almost" like the entire dataset,
that is, any (relevant) statistic
computed using a coreset will be provably close
to the quantity if computed using the entire dataset.
In this work, the goal was to evaluate whether CG type methods
can be derived for such nonsmooth problems in ML.
This paper showed that by bringing together with the
ε-subdifferential
and approximate subdifferential, there does exist a CG type algorithm
for such problems. Using this construction,
we were also able to analyze the sparsity of the solution at a given iteration,
thus computing the coreset simultaneously.
While the recent developments in computer vision
algorithms driving a range of turnkey applications
is exciting, an evolving body of works
have concurrently shown the fragility of the algorithms,
specifically with respect to random perturbations.
For instance, a state of the art vision system can
be fooled into thinking that a "Stop" sign is a "Yield" sign
simply by putting a tiny sticker on it.
I am very interested in the systematic study of the
resilience of vision algorithms and using the results or
insights obtained to design better algorithms.
