42:
a web page. Reasons why one might want to use this meta tag include advising robots not to index a very large database, web pages that are very transitory, web pages that are under development, web pages that one wishes to keep slightly more private, or the printer and mobile-friendly versions of
888:
Need to prevent parts of individual pages from being searched? If you want to exclude portions of a page from indexing, surround the text with <noindex> and </noindex> tags. This is useful, for example, if you want to exclude navigation text from
43:
pages. Since the burden of honoring a website's noindex tag lies with the author of the search robot, sometimes these tags are ignored. Also the interpretation of the noindex tag is sometimes slightly different from one search engine company to the next.
286:
It is also possible to exclude part of a Web page, for example navigation text, from being indexed rather than the whole page. There are various techniques for doing this; it is possible to use several in combination. Google's main indexing spider,
981:
303:
introduced a new <noindex> tag which prevents indexing of the content between the tags. To allow the source code to validate, <!--noindex--> alternatively can be used:
167:
The noindex directive can be restricted only to certain bots by specifying a different "name" value in the meta tag. For example, to specifically block Google's bot, specify:
128:
Possible values for the meta tag content are: "none", "all", "index", "noindex", "nofollow", and "follow". A combination of the values is also possible, for example:
659:. Inner <div>s were initially not excluded, but this may have changed. It is also unknown whether the attribute can be applied to tags other than <div>.
1003:
935:
863:
969:
533:
introduced similar functionality to the microformat into its spider. However, Yahoo!'s spider is incompatible in that it looks for the value
1041:
1013:
947:
894:
875:
820:
775:
275:
1046:
1008:
724:
39:
375:
specification with the same functionality. The Robot
Exclusion Profile looks for the attribute and value
833:
20:
1012:. Google Inc. August 23, 2012. Section: Excluding Unwanted Text from the Index. Archived from
797:
655:
2010’s iFilter excludes content inside of a <div> tag with the attribute and value
874:. 2013. Section: How do I exclude parts of my site from being searched?. Archived from
1035:
35:
372:
356:
652:
977:
288:
912:
769:
31:
943:
841:
809:
530:
300:
16:
Meta tag used to request that
Internet bots avoid indexing a web page
871:
360:
970:"Control Search Indexing (Crawling) Within a Page with Noindex"
760:
Other indexing spiders also use their own structured comments.
489:
A combination of values is also possible, for example:
291:, is not known to recognize any of these techniques.
359:also recognize the <noindex> tag, including
936:"Introducing Robots-Nocontent for Page Sections"
906:
904:
19:For the internal use on Knowledge (XXG), see
8:
1004:"Administering Crawl: Preparing for a Crawl"
821:Using meta tags to block access to your site
793:
791:
506:"robots-noindex robots-follow"
787:
984:from the original on November 4, 2017
7:
278:file can be used to block crawling.
235:Or, to block Baidu's bot, specify:
201:Or, to block Bing's bot, specify:
14:
810:About the Robots <META> tag
911:Janes, Peter (June 18, 2005).
844:. Section: <noindex> tag
823:, Google Webmasters Tools Help
1:
934:Garg, Priyank (May 2, 2007).
629:"robots-nocontent"
601:"robots-nocontent"
573:"robots-nocontent"
800:, Official W3 specification
798:Robots and the META element
744:Don't index this text.
742:<!--googleoff: all-->
154:"noindex, follow"
1063:
1042:Search engine optimization
746:<!--googleon: all-->
727:uses structured comments:
700:Don't index this text.
634:Don't index this text.
606:Don't index this text.
578:Don't index this text.
476:Don't index this text.
471:"robots-noindex"
448:Don't index this text.
443:"robots-noindex"
420:Don't index this text.
415:"robots-noindex"
339:Don't index this text.
326:Don't index this text.
299:The Russian search engine
18:
913:"Robot Exclusion Profile"
776:Robots Exclusion Standard
282:Noindexing part of a page
106:Don't index this page
729:
661:
539:
535:class="robots-nocontent"
491:
381:
305:
237:
203:
169:
130:
50:
34:requests that automated
30:value of an HTML robots
1009:Google Search Appliance
725:Google Search Appliance
719:Google Search Appliance
252:"baiduspider"
163:Bot-specific directives
47:Noindexing entire pages
377:class="robots-noindex"
371:There is a 2005 draft
895:registration required
537:and only this value:
341:<!--/noindex-->
184:"googlebot"
1016:on November 23, 2012
864:"General Search FAQ"
740:Do index this text.
337:<!--noindex-->
316:Do index this text.
974:Microsoft Developer
878:on December 8, 2021
714:Structured comments
695:"noindex"
672:Do index this text.
550:Do index this text.
392:Do index this text.
295:<noindex> tag
261:"noindex"
227:"noindex"
218:"bingbot"
193:"noindex"
92:"noindex"
950:on August 20, 2014
940:Yahoo! Search Blog
145:"robots"
83:"robots"
834:"Using HTML tags"
1054:
1026:
1025:
1023:
1021:
1000:
994:
993:
991:
989:
980:. June 7, 2010.
966:
960:
959:
957:
955:
946:. Archived from
931:
925:
924:
922:
920:
908:
899:
898:
891:
885:
883:
860:
854:
853:
851:
849:
838:webmaster → help
830:
824:
818:
812:
807:
801:
795:
756:
753:
750:
747:
743:
739:
736:
733:
709:
706:
703:
699:
696:
693:
690:
687:
684:
681:
678:
675:
671:
668:
665:
643:
640:
637:
633:
630:
627:
624:
621:
618:
615:
612:
609:
605:
602:
599:
596:
593:
590:
587:
584:
581:
577:
574:
571:
568:
565:
562:
559:
556:
553:
549:
546:
543:
520:
517:
514:
510:
507:
504:
501:
498:
495:
485:
482:
479:
475:
472:
469:
466:
463:
460:
457:
454:
451:
447:
444:
441:
438:
435:
432:
429:
426:
423:
419:
416:
413:
410:
407:
404:
401:
398:
395:
391:
388:
385:
357:indexing spiders
351:
348:
345:
342:
338:
335:
332:
329:
325:
322:
319:
315:
312:
309:
265:
262:
259:
256:
253:
250:
247:
244:
241:
231:
228:
225:
222:
219:
216:
213:
210:
207:
197:
194:
191:
188:
185:
182:
179:
176:
173:
158:
155:
152:
149:
146:
143:
140:
137:
134:
124:
121:
118:
115:
112:
109:
105:
102:
99:
96:
93:
90:
87:
84:
81:
78:
75:
72:
69:
66:
63:
60:
57:
54:
1062:
1061:
1057:
1056:
1055:
1053:
1052:
1051:
1032:
1031:
1030:
1029:
1019:
1017:
1002:
1001:
997:
987:
985:
968:
967:
963:
953:
951:
933:
932:
928:
918:
916:
910:
909:
902:
892:
881:
879:
862:
861:
857:
847:
845:
832:
831:
827:
819:
815:
808:
804:
796:
789:
784:
766:
758:
757:
754:
751:
748:
745:
741:
737:
734:
731:
721:
716:
711:
710:
707:
704:
701:
697:
694:
691:
688:
685:
682:
679:
676:
673:
669:
666:
663:
657:class="noindex"
650:
645:
644:
641:
638:
635:
631:
628:
625:
622:
619:
616:
613:
610:
607:
603:
600:
597:
594:
591:
588:
585:
582:
579:
575:
572:
569:
566:
563:
560:
557:
554:
551:
547:
544:
541:
527:
522:
521:
518:
515:
512:
508:
505:
502:
499:
496:
493:
487:
486:
483:
480:
477:
473:
470:
467:
464:
461:
458:
455:
452:
449:
445:
442:
439:
436:
433:
430:
427:
424:
421:
417:
414:
411:
408:
405:
402:
399:
396:
393:
389:
386:
383:
369:
353:
352:
349:
346:
343:
340:
336:
333:
330:
327:
323:
320:
317:
313:
310:
307:
297:
284:
272:
270:robots.txt file
267:
266:
263:
260:
257:
254:
251:
248:
245:
242:
239:
233:
232:
229:
226:
223:
220:
217:
214:
211:
208:
205:
199:
198:
195:
192:
189:
186:
183:
180:
177:
174:
171:
165:
160:
159:
156:
153:
150:
147:
144:
141:
138:
135:
132:
126:
125:
122:
119:
116:
113:
110:
107:
103:
100:
97:
94:
91:
88:
85:
82:
79:
76:
73:
70:
67:
64:
61:
58:
55:
52:
49:
24:
17:
12:
11:
5:
1060:
1058:
1050:
1049:
1047:World Wide Web
1044:
1034:
1033:
1028:
1027:
995:
961:
926:
915:. Microformats
900:
855:
825:
813:
802:
786:
785:
783:
780:
779:
778:
773:
772:link attribute
765:
762:
730:
720:
717:
715:
712:
662:
649:
646:
540:
526:
523:
492:
382:
379:in HTML tags:
368:
365:
306:
296:
293:
283:
280:
271:
268:
238:
204:
170:
164:
161:
131:
51:
48:
45:
15:
13:
10:
9:
6:
4:
3:
2:
1059:
1048:
1045:
1043:
1040:
1039:
1037:
1015:
1011:
1010:
1005:
999:
996:
983:
979:
975:
971:
965:
962:
949:
945:
941:
937:
930:
927:
914:
907:
905:
901:
896:
890:
877:
873:
869:
865:
859:
856:
843:
839:
835:
829:
826:
822:
817:
814:
811:
806:
803:
799:
794:
792:
788:
781:
777:
774:
771:
768:
767:
763:
761:
728:
726:
718:
713:
660:
658:
654:
647:
538:
536:
532:
524:
490:
380:
378:
374:
366:
364:
362:
358:
304:
302:
294:
292:
290:
281:
279:
277:
269:
236:
202:
168:
162:
129:
46:
44:
41:
37:
36:Internet bots
33:
29:
22:
1018:. Retrieved
1014:the original
1007:
998:
986:. Retrieved
973:
964:
952:. Retrieved
948:the original
939:
929:
917:. Retrieved
887:
880:. Retrieved
876:the original
867:
858:
846:. Retrieved
837:
828:
816:
805:
759:
722:
656:
651:
534:
528:
488:
376:
373:microformats
370:
354:
298:
285:
273:
234:
200:
166:
127:
27:
25:
988:November 4,
367:microformat
1036:Categories
782:References
653:SharePoint
648:SharePoint
276:robots.txt
21:WP:NOINDEX
1020:March 23,
978:Microsoft
954:March 23,
919:March 24,
889:searches.
882:March 23,
848:March 25,
529:In 2007,
289:Googlebot
982:Archived
770:Nofollow
764:See also
40:indexing
32:meta tag
331:noindex
321:noindex
255:content
221:content
187:content
148:content
86:content
28:noindex
944:Yahoo!
842:Yandex
531:Yahoo!
525:Yahoo!
355:Other
301:Yandex
38:avoid
872:Atomz
749:</
702:</
689:class
674:</
636:</
623:class
608:</
595:class
580:</
567:class
552:</
513:</
511:Text.
500:class
478:</
465:class
450:</
437:class
422:</
409:class
394:</
361:Atomz
344:</
328:</
117:</
111:title
108:</
101:title
1022:2013
990:2017
956:2013
921:2013
884:2013
868:Help
850:2013
755:>
738:>
732:<
723:The
708:>
698:>
683:<
680:>
670:>
664:<
642:>
632:>
617:<
614:>
611:span
604:>
592:span
589:<
586:>
576:>
561:<
558:>
548:>
542:<
519:>
509:>
494:<
484:>
474:>
459:<
456:>
453:span
446:>
434:span
431:<
428:>
418:>
403:<
400:>
390:>
384:<
350:>
334:>
324:>
318:<
314:>
308:<
264:>
246:name
243:meta
240:<
230:>
212:name
209:meta
206:<
196:>
178:name
175:meta
172:<
157:>
139:name
136:meta
133:<
123:>
120:head
114:>
104:>
98:<
95:>
77:name
74:meta
71:<
68:>
65:head
62:<
59:>
56:html
53:<
26:The
705:div
686:div
583:div
564:div
516:div
497:div
425:div
406:div
1038::
1006:.
976:.
972:.
942:.
938:.
903:^
886:.
870:.
866:.
840:.
836:.
790:^
363:.
274:A
1024:.
992:.
958:.
923:.
897:)
893:(
852:.
752:p
735:p
692:=
677:p
667:p
639:p
626:=
620:p
598:=
570:=
555:p
545:p
503:=
481:p
468:=
462:p
440:=
412:=
397:p
387:p
347:p
311:p
258:=
249:=
224:=
215:=
190:=
181:=
151:=
142:=
89:=
80:=
23:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.