88:
kernels allows linear projection of new samples on to the current structure of the model where novelty in new data can be easily differentiated from noise-born errors which should not result in a change to model structure. Analytical metrics for structure analysis can be used to parsimoniously grow model complexity when required or optimally prune the existing structure when processor resource limits are reached. Structure updates are also relevant when system variation is detected and the long-term memory of the model should be updated as for the
75:
used in linear adaptive filters to reduce statistical uncertainties. However because nonlinear filters typically have a much higher potential structural complexity (or higher dimensional feature space) compared to the subspace actually required, regularisation of some kind must deal with the under-determined model. Though some specific forms of parameter regularisation such as prescribed by Vapink's
96:
able to easily handle over 100,000 training examples using as little as 10kB RAM. Data sizes this large are challenging to the original formulations of support vector machines and other kernel methods, which for example relied on constrained optimisation using linear or quadratic programming techniques.
95:
Iterative gradient descent that is typically used in adaptive filters has also gained popularity in offline batch-mode support vector based machine learning because of its computational efficiency for large data set processing. Both time series and batch data processing performance is reported to be
83:
address the dimensionality problem statistically to some extent, there remain further statistical and practical issues for truly adaptive non-linear filters. Adaptive filters are often used for tracking the behaviour of a time-varying system or systems which cannot be fully modelled from the data and
87:
Where structural parameters of kernels are derived directly from data being processed (as in the above "Support Vector" approach) there are convenient opportunities for analytically robust methods of self organisation of the kernels available to the filter. The linearised feature space induced by
74:
Self organising kernel adaptive filters that use iteration to achieve convex LMS error minimisation address some of the statistical and practical issues of non-linear models that do not arise in the linear case. Regularisation is particularly important feature for non-linear models and also often
62:
Because high-dimensional feature space is linear, kernel adaptive filters can be thought of as a generalization of linear adaptive filters. As with linear adaptive filters, there are two general approaches to adapting a filter: the
55:, a kernel method can be a universal approximator for a nonlinear function. Kernel methods have the advantage of having convex loss functions, with no local minima, and of being only moderately
36:
that characterizes how far the filter deviates from ideal behavior. The adaptation process is based on learning from a sequence of signal samples and is thus an
296:
130:
286:
52:
56:
68:
76:
51:
and a nonlinear function is approximated as a sum over kernels, whose domain is the feature space. If this is done in a
291:
48:
64:
37:
80:
117:
206:
155:
146:
Liu, Weifeng; Pokharel, P.P.; Principe, J.C. (2008-02-01). "The Kernel Least-Mean-Square
Algorithm".
230:
179:
84:
structure available, hence the models may not only need to adapt parameters, but structure too.
222:
197:
Engel, Y.; Mannor, S.; Meir, R. (2004-08-01). "The kernel recursive least-squares algorithm".
171:
126:
29:
17:
214:
163:
25:
210:
159:
44:
249:
280:
183:
89:
33:
251:
Kernel
Methods and their Application to Systems Identification and Signal Processing
234:
40:. A nonlinear adaptive filter is one in which the transfer function is nonlinear.
226:
175:
218:
167:
266:
47:. In these methods, the signal is mapped to a high-dimensional linear
43:
Kernel adaptive filters implement a nonlinear transfer function using
32:
to changes in signal properties over time by minimizing an error or
116:
Weifeng Liu; José C. Principe; Simon Haykin (March 2010).
119:
Kernel
Adaptive Filtering: A Comprehensive Introduction
8:
24:is a type of nonlinear adaptive filter. An
267:"An Online Support Vector Learning Method"
111:
109:
105:
199:IEEE Transactions on Signal Processing
148:IEEE Transactions on Signal Processing
7:
297:Kernel methods for machine learning
265:Pierre Drezet; Robert F Harrison.
14:
53:reproducing kernel Hilbert space
69:recursive least squares filter
1:
28:is a filter that adapts its
313:
287:Digital signal processing
125:. Wiley. pp. 12–20.
92:case in linear filters.
65:least mean squares filter
269:. Sheffield University.
219:10.1109/TSP.2004.830985
168:10.1109/TSP.2007.907881
248:Pierre Drezet (2001).
22:kernel adaptive filter
211:2004ITSP...52.2275E
160:2008ITSP...56..543L
292:Nonlinear filters
132:978-0-470-44753-6
30:transfer function
18:signal processing
304:
271:
270:
262:
256:
255:
245:
239:
238:
205:(8): 2275–2285.
194:
188:
187:
143:
137:
136:
124:
113:
38:online algorithm
312:
311:
307:
306:
305:
303:
302:
301:
277:
276:
275:
274:
264:
263:
259:
247:
246:
242:
196:
195:
191:
145:
144:
140:
133:
122:
115:
114:
107:
102:
26:adaptive filter
12:
11:
5:
310:
308:
300:
299:
294:
289:
279:
278:
273:
272:
257:
240:
189:
154:(2): 543–554.
138:
131:
104:
103:
101:
98:
67:(LMS) and the
59:to implement.
45:kernel methods
13:
10:
9:
6:
4:
3:
2:
309:
298:
295:
293:
290:
288:
285:
284:
282:
268:
261:
258:
253:
252:
244:
241:
236:
232:
228:
224:
220:
216:
212:
208:
204:
200:
193:
190:
185:
181:
177:
173:
169:
165:
161:
157:
153:
149:
142:
139:
134:
128:
121:
120:
112:
110:
106:
99:
97:
93:
91:
90:Kalman Filter
85:
82:
78:
72:
70:
66:
60:
58:
54:
50:
49:feature space
46:
41:
39:
35:
34:loss function
31:
27:
23:
19:
260:
250:
243:
202:
198:
192:
151:
147:
141:
118:
94:
86:
73:
61:
42:
21:
15:
281:Categories
100:References
254:(Thesis).
227:1053-587X
184:206797001
176:1053-587X
235:10220028
207:Bibcode
156:Bibcode
71:(RLS).
57:complex
233:
225:
182:
174:
129:
79:&
231:S2CID
180:S2CID
123:(PDF)
223:ISSN
172:ISSN
127:ISBN
20:, a
215:doi
164:doi
81:SVM
77:SRM
16:In
283::
229:.
221:.
213:.
203:52
201:.
178:.
170:.
162:.
152:56
150:.
108:^
237:.
217::
209::
186:.
166::
158::
135:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.