159:
and was considered to be a ‘secret weapon’ that allowed
Seisint to gain market share in its data business. Equifax had an SQL-based process for predicting who would go bankrupt in the next 30 days, but it took 26 days to run the data. The first ECL implementation solved the same problem in 6 minutes.
345:
Whilst ECL is terse and LexisNexis claims that 1 line of ECL is roughly equivalent to 120 lines of C++, it still has significant support for large scale programming including data encapsulation and code re-use. The constructs available include MODULE, FUNCTION, FUNCTIONMACRO, INTERFACE, MACRO,
172:
ECL, at least in its purest form, is a declarative, data-centric language. Programs, in the strictest sense, do not exist. Rather an ECL application will specify a number of core datasets (or data values) and then the operations which are to be performed on those values.
358:
implementation, by default, most ECL constructs will execute in parallel across the hardware being used. Many of the primitives also have a LOCAL option to specify that the operation is to occur locally on each node.
462:"Evaluating use of data flow systems for large graph analysis," by A. Yoo, and I. Kaplan. Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, MTAGS, 2009
472:
261:
are defined in ECL as attribute definitions. They do not denote an action; rather a definition of a term. Thus, logically, an ECL program can be read: "bottom to top"
540:
535:
337:
ECL primitives that act upon datasets include SORT, ROLLUP, DEDUP, ITERATE, PROJECT, JOIN, NORMALIZE, DENORMALIZE, PARSE, CHOSEN, ENTH, TOPN, DISTRIBUTE
520:
143:(Enterprise Control Language) is a declarative, data-centric programming language designed in 2000 to allow a team of programmers to process
147:
across a high performance computing cluster without the programmer being involved in many of the lower level, imperative decisions.
37:
187:
Perhaps a more flavorful example would take a list of strings, sort them into order, and then return that as a result instead.
476:
132:
181:
ECL is to have succinct solutions to problems and sensible defaults. The "Hello World" program is characteristically short:
116:
515:
505:
29:
510:
155:
ECL was initially designed and developed in 2000 by David
Bayliss as an in-house productivity tool within
367:
The Hadoop Map-Reduce paradigm consists of three phases which correlate to ECL primitives as follows.
24:
446:
44:
545:
85:
329:
D is a dataset with one column labeled ‘Value’ and containing the following list of data.
164:
and then again as a major source of synergies when LexisNexis acquired ChoicePoint Inc.
529:
160:
The technology was cited as a driving force behind the acquisition of
Seisint by
450:
156:
78:
74:
70:
65:
161:
33:
405:
The records from the mapper are distributed depending upon the KEY value
195:// Datasets can also be binary, CSV, XML or externally defined structures
144:
416:
The records arriving at a particular reducer are sorted into KEY order
192:// First declare a dataset with one column containing a list of strings
124:
391:
112:
128:
90:
355:
49:
120:
500:
100:
390:
Takes a record and converts to a different format; in the
427:
The records for a particular KEY value are now combined
511:
Reed
Elsevier to acquire ChoicePoint for $ 3.6 billion
516:
Reed
Elsevier's LexisNexis Buys Seisint for $ 775 Mln
106:
96:
84:
64:
56:
43:
23:
394:case the conversion is into a key-value pair
8:
18:
298:SD is a D that has been sorted by ‘Value’
17:
369:
439:
7:
541:Data-centric programming languages
14:
536:Declarative programming languages
350:Support for Parallelism in ECL
1:
402:DISTRIBUTE(,HASH(KeyValue))
257:The statements containing a
52:, LexisNexis Risk Solutions
562:
501:Rosetta Code ECL category
384:MAPing within the MAPper
111:
473:"Acquisition of Seisint"
363:Comparison to Map-Reduce
303:
269:
189:
101:http://hpccsystems.com/
506:ECL Language Reference
424:ROLLUP(,Key,LOCAL)
168:Language constructs
57:First appeared
20:
410:SHUFFLE (Phase 2)
399:SHUFFLE (Phase 1)
387:PROJECT/TRANSFORM
431:
430:
373:Hadoop Name/Term
341:ECL encapsulation
138:
137:
66:Typing discipline
553:
488:
487:
485:
484:
475:. Archived from
469:
463:
460:
454:
444:
370:
325:
322:
319:
316:
313:
310:
307:
294:
291:
288:
285:
282:
279:
276:
273:
260:
253:
250:
247:
244:
241:
238:
235:
232:
229:
226:
223:
220:
217:
214:
211:
208:
205:
202:
199:
196:
193:
21:
561:
560:
556:
555:
554:
552:
551:
550:
526:
525:
497:
492:
491:
482:
480:
471:
470:
466:
461:
457:
445:
441:
436:
376:ECL equivalent
365:
352:
346:EXPORT, SHARED
343:
335:
327:
326:
323:
320:
317:
314:
311:
308:
305:
296:
295:
292:
289:
286:
283:
280:
277:
274:
271:
267:What is an SD?
265:
258:
255:
254:
251:
248:
245:
242:
239:
236:
233:
230:
227:
224:
221:
218:
215:
212:
209:
206:
203:
200:
197:
194:
191:
185:
179:
170:
153:
12:
11:
5:
559:
557:
549:
548:
543:
538:
528:
527:
524:
523:
518:
513:
508:
503:
496:
495:External links
493:
490:
489:
464:
455:
447:A Guide to ECL
438:
437:
435:
432:
429:
428:
425:
422:
418:
417:
414:
411:
407:
406:
403:
400:
396:
395:
388:
385:
381:
380:
377:
374:
364:
361:
351:
348:
342:
339:
334:
333:ECL primitives
331:
304:
270:
263:
190:
184:'Hello World'
183:
178:
175:
169:
166:
152:
149:
136:
135:
109:
108:
104:
103:
98:
94:
93:
88:
82:
81:
68:
62:
61:
58:
54:
53:
47:
41:
40:
27:
13:
10:
9:
6:
4:
3:
2:
558:
547:
544:
542:
539:
537:
534:
533:
531:
522:
521:Reed Elsevier
519:
517:
514:
512:
509:
507:
504:
502:
499:
498:
494:
479:on 2011-06-21
478:
474:
468:
465:
459:
456:
452:
448:
443:
440:
433:
426:
423:
420:
419:
415:
413:SORT(,LOCAL)
412:
409:
408:
404:
401:
398:
397:
393:
389:
386:
383:
382:
378:
375:
372:
371:
368:
362:
360:
357:
349:
347:
340:
338:
332:
330:
302:
301:What is a D?
299:
268:
262:
188:
182:
176:
174:
167:
165:
163:
158:
150:
148:
146:
142:
134:
130:
126:
122:
118:
114:
110:
107:Influenced by
105:
102:
99:
95:
92:
89:
87:
83:
80:
76:
72:
69:
67:
63:
59:
55:
51:
50:HPCC Systems®
48:
46:
42:
39:
35:
31:
28:
26:
22:
16:
481:. Retrieved
477:the original
467:
458:
442:
366:
353:
344:
336:
328:
300:
297:
266:
256:
186:
180:
171:
154:
140:
139:
38:data-centric
15:
451:Lexis-Nexis
264:OUTPUT(SD)
177:Hello world
157:Seisint Inc
30:declarative
530:Categories
483:2011-03-24
434:References
162:LexisNexis
34:structured
379:Comments
45:Developer
546:Big data
145:big data
25:Paradigm
421:REDUCE
354:In the
312:DATASET
204:DATASET
151:History
133:Clarion
125:Snobol4
97:Website
392:Hadoop
318:STRING
243:output
210:STRING
117:Pascal
113:Prolog
75:strong
71:static
321:Value
290:Value
237:Value
213:Value
91:Linux
356:HPCC
324:;});
278:SORT
225:SORT
216:;});
79:safe
60:2000
315:(,{
207:(,{
141:ECL
129:C++
121:SQL
19:ECL
532::
449:,
309::=
293:);
275::=
272:SD
259::=
249:SD
240:);
222::=
219:SD
201::=
131:,
127:,
123:,
119:,
115:,
86:OS
77:,
73:,
36:,
32:,
486:.
453:.
306:D
287:,
284:D
281:(
252:)
246:(
234:,
231:D
228:(
198:D
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.