{"id":1389,"date":"2011-04-11T19:10:54","date_gmt":"2011-04-11T23:10:54","guid":{"rendered":"http:\/\/scientopia.org\/blogs\/goodmath\/?p=1389"},"modified":"2011-04-11T19:10:54","modified_gmt":"2011-04-11T23:10:54","slug":"nondeterminism-in-finite-state-automata","status":"publish","type":"post","link":"http:\/\/www.goodmath.org\/blog\/2011\/04\/11\/nondeterminism-in-finite-state-automata\/","title":{"rendered":"Nondeterminism in Finite State Automata"},"content":{"rendered":"<p> In the last post, I mentioned the fact that regular expressions specify the same set of languages as regular grammars, and that finite state machines are the computational device that can recognize those languages.<\/p>\n<p> It&#8217;s even pretty easy to describe how to convert from regular expressions to FSMs. But before we do that, to make it a bit easier, we&#8217;ll extend our finite state machines. Doing that is interesting in itself: What we&#8217;re going to do is create <em>non-deterministic<\/em> finite state machines &#8211; NFA (for nondeterministic finite automata) for short.<\/p>\n<p><!--more--><\/p>\n<p> NFA are interesting things. They&#8217;re a variation on the deterministic state machines (DFA) that we looked at before. An NFA is pretty similar to the DFA, except that the machine is allowed  to make <em>choices<\/em>.<\/p>\n<p> In the formulation that I&#8217;m using, there are two key differences between DFAs and NFAs:<\/p>\n<dl>\n<dt>Multiple Transitions\/Choices<\/dt>\n<dd> There can be more than one transition from a given state on a given input symbol.<\/dd>\n<dt>Empty transitions<\/dt>\n<dd> Two states in an NFA can be connected by an edge labelled <img src='http:\/\/l.wordpress.com\/latex.php?latex=epsilon&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='epsilon' style='vertical-align:1%' class='tex' alt='epsilon' \/>. This is called a <em>null transition<\/em>. <\/dd>\n<\/dl>\n<p> The non-deterministic machine runs in pretty much the same way as a deterministic machine <em>except<\/em>:<\/p>\n<ul>\n<li> If there is a null transition from a state, then a machine in that state can take the transition any time that they want, without consuming any inputs.<\/li>\n<li> If there is more than one transition from a state on a particular machine, the machine can choose any of the available transitions.<\/li>\n<\/ul>\n<p> A non-deterministic machine accepts an input string if there is <em>any<\/em> possible way of running the machine on a string that ends in a final state. For example:<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/scientopia.org\/blogs\/goodmath\/files\/2011\/04\/nfa.png\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/scientopia.org\/blogs\/goodmath\/files\/2011\/04\/nfa.png?resize=313%2C249\" alt=\"\" title=\"nfa\" width=\"313\" height=\"249\" class=\"alignright size-full wp-image-1390\" srcset=\"https:\/\/i0.wp.com\/www.goodmath.org\/blog\/wp-content\/uploads\/2011\/04\/nfa.png?w=313 313w, https:\/\/i0.wp.com\/www.goodmath.org\/blog\/wp-content\/uploads\/2011\/04\/nfa.png?resize=300%2C238 300w\" sizes=\"auto, (max-width: 313px) 100vw, 313px\" \/><\/a><\/p>\n<p> This machine contains one transition labelled with an <img src='http:\/\/l.wordpress.com\/latex.php?latex=epsilon&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='epsilon' style='vertical-align:1%' class='tex' alt='epsilon' \/>, meaning that the transition can be taken from <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_0&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_0' style='vertical-align:1%' class='tex' alt='S_0' \/> without consuming any input.<\/p>\n<p> So &#8211; the interesting question that NFAs bring up is: does changing the machine this way do anything? Does it give us any more power? Unfortunately, no. Given a non-deterministic FSM, we can easily convert it to a deterministic FSM. <\/p>\n<ul>\n<li> The DFA will have one state For each member of the <em>powerset<\/em>     of the states of the NFA.<\/li>\n<li> The initial state of the DFA is the same as the initial state of the    NFA.<\/li>\n<li> A state in the DFA is final if any of its members is a final state in the NFA.<\/li>\n<li> For each state <img src='http:\/\/l.wordpress.com\/latex.php?latex=S%3D%7B%20s_1%2C%20...%2C%20s_n%20%7D&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S={ s_1, ..., s_n }' style='vertical-align:1%' class='tex' alt='S={ s_1, ..., s_n }' \/> and each input symbol <img src='http:\/\/l.wordpress.com\/latex.php?latex=k&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='k' style='vertical-align:1%' class='tex' alt='k' \/>, there is a transition to state <img src='http:\/\/l.wordpress.com\/latex.php?latex=T%3D%7B%20t_1%2C%20...%2C%20t_n%7D&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='T={ t_1, ..., t_n}' style='vertical-align:1%' class='tex' alt='T={ t_1, ..., t_n}' \/> if there are sequences of empty transitions surrounding a single non-empty transition for <img src='http:\/\/l.wordpress.com\/latex.php?latex=k&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='k' style='vertical-align:1%' class='tex' alt='k' \/> from members of <img src='http:\/\/l.wordpress.com\/latex.php?latex=S&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S' style='vertical-align:1%' class='tex' alt='S' \/> to members of <img src='http:\/\/l.wordpress.com\/latex.php?latex=T&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='T' style='vertical-align:1%' class='tex' alt='T' \/> in the NFA.<\/li>\n<\/ul>\n<p> Basically, what that means is that you can create a DFA which superimposes every possible state that the NFA could be in. After any input string <img src='http:\/\/l.wordpress.com\/latex.php?latex=s&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='s' style='vertical-align:1%' class='tex' alt='s' \/> has been consumed, the state of the DFA is the set of states that the NFA could have been in after seeing that string.<\/p>\n<p> So, for example, the NFA shown above can be converted into the following DFA:<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/scientopia.org\/blogs\/goodmath\/files\/2011\/04\/dfa.png\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/scientopia.org\/blogs\/goodmath\/files\/2011\/04\/dfa.png?resize=476%2C263\" alt=\"\" title=\"dfa\" width=\"476\" height=\"263\" class=\"alignright size-full wp-image-1391\" srcset=\"https:\/\/i0.wp.com\/www.goodmath.org\/blog\/wp-content\/uploads\/2011\/04\/dfa.png?w=476 476w, https:\/\/i0.wp.com\/www.goodmath.org\/blog\/wp-content\/uploads\/2011\/04\/dfa.png?resize=300%2C165 300w\" sizes=\"auto, (max-width: 476px) 100vw, 476px\" \/><\/a><\/p>\n<p> We start with <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_0&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_0' style='vertical-align:1%' class='tex' alt='S_0' \/>. From <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_0&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_0' style='vertical-align:1%' class='tex' alt='S_0' \/>, if we see an &#8220;a&#8221;, there are three things that could happen. Either we&#8217;d take the transition to <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_2&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_2' style='vertical-align:1%' class='tex' alt='S_2' \/>, or we&#8217;d take the empty transition to <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_1&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_1' style='vertical-align:1%' class='tex' alt='S_1' \/>, and then take the transition from <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_1&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_1' style='vertical-align:1%' class='tex' alt='S_1' \/> to <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_1&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_1' style='vertical-align:1%' class='tex' alt='S_1' \/>, or we&#8217;d take the empty transition to <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_1&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_1' style='vertical-align:1%' class='tex' alt='S_1' \/>, and then take the a-transition to <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_f&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_f' style='vertical-align:1%' class='tex' alt='S_f' \/>. So in the DFA, we&#8217;d have an a-transition from <img src='http:\/\/l.wordpress.com\/latex.php?latex=%7BS_0%7D&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='{S_0}' style='vertical-align:1%' class='tex' alt='{S_0}' \/> to <img src='http:\/\/l.wordpress.com\/latex.php?latex=%7BS_1%2C%20S_2%2C%20S_f%20%7D&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='{S_1, S_2, S_f }' style='vertical-align:1%' class='tex' alt='{S_1, S_2, S_f }' \/>. <\/p>\n<p> If we saw a <img src='http:\/\/l.wordpress.com\/latex.php?latex=b&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='b' style='vertical-align:1%' class='tex' alt='b' \/>, there are no transitions shown. By convention, we assume some &#8220;invisible&#8221; states. Any time that there&#8217;s no transition listed for a given input symbol from a state, we assume that there&#8217;s a transition from the state to &#8220;error&#8221;. But we can use that assumption in the DFA as well as the NFA &#8211; so we just won&#8217;t show the error state or transitions &#8211; that makes our machines much easier to read.<\/p>\n<p> Analyzing state <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_0&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_0' style='vertical-align:1%' class='tex' alt='S_0' \/> has given us state <img src='http:\/\/l.wordpress.com\/latex.php?latex=%7BS_1%2C%20S_2%2C%20S_f%20%7D&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='{S_1, S_2, S_f }' style='vertical-align:1%' class='tex' alt='{S_1, S_2, S_f }' \/>. If we see an input of a, that could bring us to <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_1&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_1' style='vertical-align:1%' class='tex' alt='S_1' \/> or <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_f&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_f' style='vertical-align:1%' class='tex' alt='S_f' \/> (if the NFA was in state <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_1&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_1' style='vertical-align:1%' class='tex' alt='S_1' \/>); or it could bring us to state <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_f&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_f' style='vertical-align:1%' class='tex' alt='S_f' \/> (if the NFA was in state <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_2&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_2' style='vertical-align:1%' class='tex' alt='S_2' \/>.) So for an input of a, we&#8217;d put a transition to state <img src='http:\/\/l.wordpress.com\/latex.php?latex=%7B%20S_1%2C%20S_f%7D&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='{ S_1, S_f}' style='vertical-align:1%' class='tex' alt='{ S_1, S_f}' \/>. If we saw an input\tof &#8220;b&#8221;, we could either go from <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_2&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_2' style='vertical-align:1%' class='tex' alt='S_2' \/> to <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_2&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_2' style='vertical-align:1%' class='tex' alt='S_2' \/>, or\tfrom <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_2&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_2' style='vertical-align:1%' class='tex' alt='S_2' \/> to <img src='http:\/\/l.wordpress.com\/latex.php?latex=S_f&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='S_f' style='vertical-align:1%' class='tex' alt='S_f' \/>. So for b, we&#8217;d have a transition to <img src='http:\/\/l.wordpress.com\/latex.php?latex=%7B%20S_2%2C%20S_f%20%7D&#038;bg=FFFFFF&#038;fg=000000&#038;s=0' title='{ S_2, S_f }' style='vertical-align:1%' class='tex' alt='{ S_2, S_f }' \/>.<\/p>\n<p> And so on &#8211; the final result being:<\/p>\n<p> What does this <em>mean?<\/em> <\/p>\n<p> Well, a NFA isn&#8217;t more powerful than a deterministic machine (a <em>DFA<\/em>)- there&#8217;s nothing that an NFA can do that a DFA can&#8217;t. But the NFA can do it with a <em>lot<\/em> fewer states. The DFA is a lot more complicated; it can have exponentially more states. But for an NFA, there&#8217;s a corresponding DFA that can recognize the same language.<\/p>\n<p> Most importantly&#8230; it means that with a fixed, finite amount of state, non-determinism doesn&#8217;t buy you any computational power. With a finite state machine, non-determinism can&#8217;t doesn&#8217;t even make it possible to do anything faster! The only benefit that non-determinism has at this level is space: the amount of state that you need (that is, the number of states in the machine) can be smaller with a non-deterministic machine.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the last post, I mentioned the fact that regular expressions specify the same set of languages as regular grammars, and that finite state machines are the computational device that can recognize those languages. It&#8217;s even pretty easy to describe how to convert from regular expressions to FSMs. But before we do that, to make [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[79],"tags":[107,132,156,204],"class_list":["post-1389","post","type-post","status-publish","format-standard","hentry","category-computation","tag-automata","tag-computation-2","tag-finite-state-machines","tag-non-determinism"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4lzZS-mp","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts\/1389","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/comments?post=1389"}],"version-history":[{"count":0,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts\/1389\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/media?parent=1389"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/categories?post=1389"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/tags?post=1389"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}