170:
1497:
583:
1477:
26:
352:
Horovod, a distributed training framework for TensorFlow, Keras and PyTorch, improves speed, scale and resource allocation in machine learning training activities. Uber uses
Horovod for self-driving vehicles, fraud detection, and trip forecasting. It is also being used by Alibaba, Amazon and
1561:
1371:
1213:
619:
408:
1556:
1546:
1566:
1541:
311:
729:
612:
1402:
428:
417:
401:
226:
1503:
1054:
791:
1551:
1315:
942:
749:
605:
1270:
1536:
394:
1457:
1397:
995:
217:(LF AI). Horovod has the goal of improving the speed, scale, and resource allocation when training a machine learning model.
990:
679:
96:
1432:
786:
739:
734:
516:
1483:
779:
705:
481:
67:
1107:
1042:
643:
231:
1508:
1366:
1005:
836:
659:
1407:
664:
511:
1452:
1437:
1090:
1085:
985:
853:
634:
136:
1531:
1412:
1172:
891:
886:
586:
1442:
1427:
1392:
1080:
980:
848:
1310:
1462:
1417:
863:
808:
654:
649:
536:
438:
187:
169:
1037:
1015:
764:
759:
717:
669:
562:
131:
1476:
1422:
1000:
829:
190:
149:
111:
1488:
1280:
932:
803:
796:
552:
506:
1233:
1223:
1030:
824:
774:
769:
712:
700:
456:
214:
144:
1346:
1290:
1112:
754:
674:
557:
257:
1320:
1285:
1275:
1100:
858:
684:
461:
1525:
1265:
1245:
1162:
841:
194:
183:
1351:
1182:
451:
446:
210:
1447:
1218:
1127:
1122:
744:
722:
312:"Uber brings Horovod project for distributed deep learning to Linux Foundation"
1341:
1300:
1295:
1208:
1117:
1025:
937:
917:
544:
501:
285:
236:
198:
33:
1336:
1305:
1203:
1047:
1010:
947:
901:
896:
881:
526:
466:
597:
1238:
1070:
521:
386:
25:
1361:
1198:
1152:
1075:
975:
970:
922:
547:
496:
206:
124:
1376:
1356:
1228:
1020:
567:
491:
486:
381:
377:
336:
100:
1177:
1157:
1147:
1142:
1137:
1132:
1095:
927:
476:
202:
120:
116:
1167:
471:
104:
38:
601:
390:
370:
160:
1385:
1329:
1258:
1191:
1063:
963:
956:
910:
874:
817:
693:
633:
535:
437:
155:
143:
130:
110:
92:
66:
44:
32:
1562:Python (programming language) scientific libraries
613:
402:
8:
18:
960:
620:
606:
598:
409:
395:
387:
168:
24:
17:
248:
7:
1458:Generative adversarial network (GAN)
227:Comparison of deep learning software
1557:Open-source artificial intelligence
310:Khari Johnson (December 13, 2018).
1547:Free software programmed in Python
258:"Release v0.9.0 · horovod/horovod"
14:
1567:Software using the Apache license
1496:
1495:
1475:
582:
581:
1542:Free software programmed in C++
256:Alex Sergeev (August 9, 2017).
73:v0.28.1 / June 12, 2023
1408:Recurrent neural network (RNN)
1398:Differentiable neural computer
213:. Horovod is hosted under the
1:
1453:Variational autoencoder (VAE)
1413:Long short-term memory (LSTM)
680:Computational learning theory
1433:Convolutional neural network
286:"Releases · horovod/horovod"
1428:Multilayer perceptron (MLP)
482:Microsoft Cognitive Toolkit
1583:
1504:Artificial neural networks
1418:Gated recurrent unit (GRU)
644:Differentiable programming
232:Differentiable programming
1552:Free statistical software
1471:
837:Artificial neural network
660:Automatic differentiation
577:
424:
88:
62:
23:
665:Neuromorphic engineering
628:Differentiable computing
341:Linux Foundation - LF AI
1438:Residual neural network
854:Artificial Intelligence
137:Artificial intelligence
75:; 15 months ago
1537:Deep learning software
418:Deep learning software
1393:Neural Turing machine
981:Human image synthesis
570:Deep Learning Toolbox
50:; 7 years ago
1484:Computer programming
1463:Graph neural network
1038:Text-to-video models
1016:Text-to-image models
864:Large language model
849:Scientific computing
655:Statistical manifold
650:Information geometry
830:In-context learning
670:Pattern recognition
563:Wolfram Mathematica
215:Linux Foundation AI
48:August 9, 2017
20:
1423:Echo state network
1311:Jürgen Schmidhuber
1006:Facial recognition
1001:Speech recognition
911:Software libraries
337:"Projects - LF AI"
191:software framework
150:Apache License 2.0
1519:
1518:
1281:Stephen Grossberg
1254:
1253:
595:
594:
177:
176:
1574:
1509:Machine learning
1499:
1498:
1479:
1234:Action selection
1224:Self-driving car
1031:Stable Diffusion
996:Speech synthesis
961:
825:Machine learning
701:Gradient descent
622:
615:
608:
599:
585:
584:
411:
404:
397:
388:
374:
373:
371:Official website
356:
355:
349:
347:
333:
327:
326:
324:
322:
307:
301:
300:
298:
296:
282:
276:
275:
270:
268:
253:
193:for distributed
173:
172:
165:
162:
83:
81:
76:
58:
56:
51:
28:
21:
1582:
1581:
1577:
1576:
1575:
1573:
1572:
1571:
1522:
1521:
1520:
1515:
1467:
1381:
1347:Google DeepMind
1325:
1291:Geoffrey Hinton
1250:
1187:
1113:Project Debater
1059:
957:Implementations
952:
906:
870:
813:
755:Backpropagation
689:
675:Tensor calculus
629:
626:
596:
591:
573:
558:Neural Designer
531:
433:
420:
415:
369:
368:
365:
360:
359:
345:
343:
335:
334:
330:
320:
318:
309:
308:
304:
294:
292:
284:
283:
279:
273:Initial release
266:
264:
255:
254:
250:
245:
223:
197:training using
167:
159:
84:
79:
77:
74:
54:
52:
49:
45:Initial release
12:
11:
5:
1580:
1578:
1570:
1569:
1564:
1559:
1554:
1549:
1544:
1539:
1534:
1524:
1523:
1517:
1516:
1514:
1513:
1512:
1511:
1506:
1493:
1492:
1491:
1486:
1472:
1469:
1468:
1466:
1465:
1460:
1455:
1450:
1445:
1440:
1435:
1430:
1425:
1420:
1415:
1410:
1405:
1400:
1395:
1389:
1387:
1383:
1382:
1380:
1379:
1374:
1369:
1364:
1359:
1354:
1349:
1344:
1339:
1333:
1331:
1327:
1326:
1324:
1323:
1321:Ilya Sutskever
1318:
1313:
1308:
1303:
1298:
1293:
1288:
1286:Demis Hassabis
1283:
1278:
1276:Ian Goodfellow
1273:
1268:
1262:
1260:
1256:
1255:
1252:
1251:
1249:
1248:
1243:
1242:
1241:
1231:
1226:
1221:
1216:
1211:
1206:
1201:
1195:
1193:
1189:
1188:
1186:
1185:
1180:
1175:
1170:
1165:
1160:
1155:
1150:
1145:
1140:
1135:
1130:
1125:
1120:
1115:
1110:
1105:
1104:
1103:
1093:
1088:
1083:
1078:
1073:
1067:
1065:
1061:
1060:
1058:
1057:
1052:
1051:
1050:
1045:
1035:
1034:
1033:
1028:
1023:
1013:
1008:
1003:
998:
993:
988:
983:
978:
973:
967:
965:
958:
954:
953:
951:
950:
945:
940:
935:
930:
925:
920:
914:
912:
908:
907:
905:
904:
899:
894:
889:
884:
878:
876:
872:
871:
869:
868:
867:
866:
859:Language model
856:
851:
846:
845:
844:
834:
833:
832:
821:
819:
815:
814:
812:
811:
809:Autoregression
806:
801:
800:
799:
789:
787:Regularization
784:
783:
782:
777:
772:
762:
757:
752:
750:Loss functions
747:
742:
737:
732:
727:
726:
725:
715:
710:
709:
708:
697:
695:
691:
690:
688:
687:
685:Inductive bias
682:
677:
672:
667:
662:
657:
652:
647:
639:
637:
631:
630:
627:
625:
624:
617:
610:
602:
593:
592:
590:
589:
578:
575:
574:
572:
571:
565:
560:
555:
550:
541:
539:
533:
532:
530:
529:
524:
519:
514:
509:
504:
499:
494:
489:
484:
479:
474:
469:
464:
462:Deeplearning4j
459:
454:
449:
443:
441:
435:
434:
432:
431:
425:
422:
421:
416:
414:
413:
406:
399:
391:
385:
384:
375:
364:
363:External links
361:
358:
357:
328:
302:
277:
247:
246:
244:
241:
240:
239:
234:
229:
222:
219:
175:
174:
157:
153:
152:
147:
141:
140:
134:
128:
127:
114:
108:
107:
94:
90:
89:
86:
85:
72:
70:
68:Stable release
64:
63:
60:
59:
46:
42:
41:
36:
30:
29:
13:
10:
9:
6:
4:
3:
2:
1579:
1568:
1565:
1563:
1560:
1558:
1555:
1553:
1550:
1548:
1545:
1543:
1540:
1538:
1535:
1533:
1532:2017 software
1530:
1529:
1527:
1510:
1507:
1505:
1502:
1501:
1494:
1490:
1487:
1485:
1482:
1481:
1478:
1474:
1473:
1470:
1464:
1461:
1459:
1456:
1454:
1451:
1449:
1446:
1444:
1441:
1439:
1436:
1434:
1431:
1429:
1426:
1424:
1421:
1419:
1416:
1414:
1411:
1409:
1406:
1404:
1401:
1399:
1396:
1394:
1391:
1390:
1388:
1386:Architectures
1384:
1378:
1375:
1373:
1370:
1368:
1365:
1363:
1360:
1358:
1355:
1353:
1350:
1348:
1345:
1343:
1340:
1338:
1335:
1334:
1332:
1330:Organizations
1328:
1322:
1319:
1317:
1314:
1312:
1309:
1307:
1304:
1302:
1299:
1297:
1294:
1292:
1289:
1287:
1284:
1282:
1279:
1277:
1274:
1272:
1269:
1267:
1266:Yoshua Bengio
1264:
1263:
1261:
1257:
1247:
1246:Robot control
1244:
1240:
1237:
1236:
1235:
1232:
1230:
1227:
1225:
1222:
1220:
1217:
1215:
1212:
1210:
1207:
1205:
1202:
1200:
1197:
1196:
1194:
1190:
1184:
1181:
1179:
1176:
1174:
1171:
1169:
1166:
1164:
1163:Chinchilla AI
1161:
1159:
1156:
1154:
1151:
1149:
1146:
1144:
1141:
1139:
1136:
1134:
1131:
1129:
1126:
1124:
1121:
1119:
1116:
1114:
1111:
1109:
1106:
1102:
1099:
1098:
1097:
1094:
1092:
1089:
1087:
1084:
1082:
1079:
1077:
1074:
1072:
1069:
1068:
1066:
1062:
1056:
1053:
1049:
1046:
1044:
1041:
1040:
1039:
1036:
1032:
1029:
1027:
1024:
1022:
1019:
1018:
1017:
1014:
1012:
1009:
1007:
1004:
1002:
999:
997:
994:
992:
989:
987:
984:
982:
979:
977:
974:
972:
969:
968:
966:
962:
959:
955:
949:
946:
944:
941:
939:
936:
934:
931:
929:
926:
924:
921:
919:
916:
915:
913:
909:
903:
900:
898:
895:
893:
890:
888:
885:
883:
880:
879:
877:
873:
865:
862:
861:
860:
857:
855:
852:
850:
847:
843:
842:Deep learning
840:
839:
838:
835:
831:
828:
827:
826:
823:
822:
820:
816:
810:
807:
805:
802:
798:
795:
794:
793:
790:
788:
785:
781:
778:
776:
773:
771:
768:
767:
766:
763:
761:
758:
756:
753:
751:
748:
746:
743:
741:
738:
736:
733:
731:
730:Hallucination
728:
724:
721:
720:
719:
716:
714:
711:
707:
704:
703:
702:
699:
698:
696:
692:
686:
683:
681:
678:
676:
673:
671:
668:
666:
663:
661:
658:
656:
653:
651:
648:
646:
645:
641:
640:
638:
636:
632:
623:
618:
616:
611:
609:
604:
603:
600:
588:
580:
579:
576:
569:
566:
564:
561:
559:
556:
554:
551:
549:
546:
543:
542:
540:
538:
534:
528:
525:
523:
520:
518:
515:
513:
510:
508:
505:
503:
500:
498:
495:
493:
490:
488:
485:
483:
480:
478:
475:
473:
470:
468:
465:
463:
460:
458:
455:
453:
450:
448:
445:
444:
442:
440:
436:
430:
427:
426:
423:
419:
412:
407:
405:
400:
398:
393:
392:
389:
383:
379:
376:
372:
367:
366:
362:
354:
342:
338:
332:
329:
317:
313:
306:
303:
291:
287:
281:
278:
274:
263:
259:
252:
249:
242:
238:
235:
233:
230:
228:
225:
224:
220:
218:
216:
212:
208:
204:
200:
196:
195:deep learning
192:
189:
185:
181:
171:
164:
158:
154:
151:
148:
146:
142:
138:
135:
133:
129:
126:
122:
118:
115:
113:
109:
106:
102:
98:
95:
91:
87:
71:
69:
65:
61:
47:
43:
40:
37:
35:
31:
27:
22:
16:
1352:Hugging Face
1316:David Silver
964:Audio–visual
818:Applications
797:Augmentation
642:
452:Apache SINGA
447:Apache MXNet
351:
344:. Retrieved
340:
331:
319:. Retrieved
315:
305:
293:. Retrieved
289:
280:
272:
265:. Retrieved
261:
251:
211:Apache MXNet
179:
178:
34:Developer(s)
15:
1500:Categories
1448:Autoencoder
1403:Transformer
1271:Alex Graves
1219:OpenAI Five
1123:IBM Watsonx
745:Convolution
723:Overfitting
537:Proprietary
439:Open source
316:VentureBeat
188:open-source
1526:Categories
1489:Technology
1342:EleutherAI
1301:Fei-Fei Li
1296:Yann LeCun
1209:Q-learning
1192:Decisional
1118:IBM Watson
1026:Midjourney
918:TensorFlow
765:Activation
718:Regression
713:Clustering
553:IBM Watson
502:TensorFlow
429:Comparison
243:References
237:All-Reduce
199:TensorFlow
93:Written in
80:2023-06-12
55:2017-08-09
1372:MIT CSAIL
1337:Anthropic
1306:Andrew Ng
1204:AlphaZero
1048:VideoPoet
1011:AlphaFold
948:MindSpore
902:SpiNNaker
897:Memristor
804:Diffusion
780:Rectifier
760:Batchnorm
740:Attention
735:Adversary
527:MindSpore
467:DeepSpeed
139:ecosystem
1480:Portals
1239:Auto-GPT
1071:Word2vec
875:Hardware
792:Datasets
694:Concepts
587:Category
522:OpenVINO
295:July 11,
221:See also
112:Platform
1362:Meta AI
1199:AlphaGo
1183:PanGu-Σ
1153:ChatGPT
1128:Granite
1076:Seq2seq
1055:Whisper
976:WaveNet
971:AlexNet
943:Flux.jl
923:PyTorch
775:Sigmoid
770:Softmax
635:General
548:Core ML
497:PyTorch
378:horovod
353:NVIDIA.
346:July 9,
321:July 9,
290:horovod
267:July 9,
262:horovod
207:PyTorch
180:Horovod
161:horovod
156:Website
145:License
125:Windows
78: (
53: (
19:Horovod
1377:Huawei
1357:OpenAI
1259:People
1229:MuZero
1091:Gemini
1086:Claude
1021:DALL-E
933:Theano
568:MATLAB
507:Theano
492:OpenNN
487:ML.NET
382:GitHub
209:, and
166:
97:Python
1443:Mamba
1214:SARSA
1178:LLaMA
1173:BLOOM
1158:GPT-J
1148:GPT-4
1143:GPT-3
1138:GPT-2
1133:GPT-1
1096:LaMDA
928:Keras
545:Apple
512:Torch
477:Keras
457:Caffe
203:Keras
182:is a
121:macOS
117:Linux
1367:Mila
1168:PaLM
1101:Bard
1081:BERT
1064:Text
1043:Sora
517:ONNX
472:Dlib
348:2020
323:2020
297:2023
269:2020
186:and
184:free
132:Type
105:CUDA
39:Uber
1108:NMT
991:OCR
986:HWR
938:JAX
892:VPU
887:TPU
882:IPU
706:SGD
380:on
163:.ai
101:C++
1528::
350:.
339:.
314:.
288:.
271:.
260:.
205:,
201:,
123:,
119:,
103:,
99:,
621:e
614:t
607:v
410:e
403:t
396:v
325:.
299:.
82:)
57:)
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
