Transfer Learning
[deep-learning transfer-learning 
Transfer learning transfer your knowledge learned from one domain/task to the other domain/task. For example, the feature extractor of CNN based image classifier can be transfered to CNN based object dection or image segmentation task.
Here we will briefly introduce some typical transfer learning methods.
Instance based Transfer Learning
Instance based Transfer Learning resample the data according to the frequence of the data in the target domain. Thus this method is limited to scenario where source domain and target domain is similar.
Feature based Transfer Learning
Feature based Transfer Learning tries reduce the difference of the source domain and target domain, by mapping the feature from source domain to target domain, or mapping the feature from source domain and target domain to an unified domain.
Parameter/Model based Transfer Learning
Parameter/Model based Transfer Learning tries to identify whether some parameters can be shared between source domain and target domain.
Relation Based Transfer Learning
Relation Based Transfer Learning assumes the samples in source domain shares the similar relationship as the samples in target domain. It is a less studied area.
Distribution Adaptation
Distribution Adaptation applies some transform to make the distribution of data in target domain similar to that of source domain. There are several methods:
- Marginal Distribution Adaptation \(DISTANCE(Ds,Dt)\approx\lVert P(X_s)-P(X_t)\Vert\)
- Conditional Distribution Adaptation \(DISTANCE(Ds,Dt)\approx\lVert P(y_s|X_s)-P(y_t|X_t)\Vert\)
- Joint Distribution Adaptation \(DISTANCE(Ds,Dt)\approx\lVert P(X_s)-P(X_t)\Vert-\lVert P(y_s|X_s)-P(y_t|X_t)\Vert\)
- Subspace Alignment
- Manifold Method: try to find a sequence of transform from source domain to target domain on a manifold
