{"id":92,"date":"2008-04-09T23:09:11","date_gmt":"2008-04-09T23:09:11","guid":{"rendered":"http:\/\/davidgerard.co.uk\/notes\/2008\/04\/09\/regular-expressions-to-ebnf\/"},"modified":"2008-04-09T23:12:36","modified_gmt":"2008-04-09T23:12:36","slug":"regular-expressions-to-ebnf","status":"publish","type":"post","link":"https:\/\/davidgerard.co.uk\/notes\/2008\/04\/09\/regular-expressions-to-ebnf\/","title":{"rendered":"Regular expressions to EBNF?"},"content":{"rendered":"

Last Thursday at London.PM<\/a>, I got asked a lot why MediaWiki<\/a> wikitext<\/a> doesn’t have a WYSIWYG editor<\/a>. The answer is that a WYSIWYG editor would need to know wikitext grammar<\/a>, and there is no defined grammar<\/i>. The MediaWiki “parser” is not actually a parser — it’s a twisty series<\/a> of regular expressions<\/a> (PHP’s version of PCRE<\/a>s).<\/p>\n

So any grammar effort (and several What You See Is All You Get editors — others just forget wikitext and write HTML) requires reverse-engineering that, and lots of people have tried<\/a> and gotten 90% of the way<\/a> before stalling. It doesn’t help that wikitext is (I’m told) provably impossible to just put into a single lump of EBNF<\/a>.<\/p>\n

The goal is to replace the twisty series of regexps<\/a> with something generated from a grammar. Tim Starling<\/a> has said, more or less: “We can’t change wikitext. Go away and write something that (a) covers almost all of it (b) is comparably fast in PHP.”<\/i> Harsh, but fair.<\/p>\n

It occurred to me that there must exist tools to convert regexps into EBNF. And that if we can get it into even a few disparate lumps of hideous EBNF, there should be tools to take those and simplify them somewhat. (Presumably with steps to say what given bits mean.) Or possibly things other than EBNF<\/a>, just as long as the result is parseable.<\/p>\n

I am not (even slightly) a computer scientist, but many of you are. Does anyone have any ideas on this? Or pointers to anyone having done anything even remotely similar? Or knowledgeable friends they could point this query at?<\/p>\n

The other approach is parserTests.php. Running maintenance scripts<\/a>, the scripts (look for parserTests), the list of tests<\/a>. A “parser” will be anything that passes the unit tests.<\/p>\n","protected":false},"excerpt":{"rendered":"

Last Thursday at London.PM, I got asked a lot why MediaWiki wikitext doesn’t have a WYSIWYG editor. The answer is that a WYSIWYG editor would need to know wikitext grammar, and there is no defined grammar. The MediaWiki “parser” is not actually a parser — it’s a twisty series of regular expressions (PHP’s version of … Continue reading “Regular expressions to EBNF?”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[4],"tags":[],"class_list":["post-92","post","type-post","status-publish","format-standard","hentry","category-wiki"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4FmVR-1u","_links":{"self":[{"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/posts\/92","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/comments?post=92"}],"version-history":[{"count":0,"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/posts\/92\/revisions"}],"wp:attachment":[{"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/media?parent=92"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/categories?post=92"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/davidgerard.co.uk\/notes\/wp-json\/wp\/v2\/tags?post=92"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}