Macro char `#` doubles when made letter?

Clash Royale CLAN TAG#URR8PPP
up vote
8
down vote
favorite
If I make a command that takes one argument, and the argument should be allowed to take the unescaped default macro char: # as an argument and then treat it as a letter:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
The above works except it prints the following:One hash: ##, not two. Is this expected behaviour? How could one fix this?
macros catcodes
add a comment |Â
up vote
8
down vote
favorite
If I make a command that takes one argument, and the argument should be allowed to take the unescaped default macro char: # as an argument and then treat it as a letter:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
The above works except it prints the following:One hash: ##, not two. Is this expected behaviour? How could one fix this?
macros catcodes
1
When the macroallowhashis called, the category code of#is still 6, so it is doubled when the argument is absorbed. With a laterscantokensyou get two#characters with category code 11 (12 would be a better choice). By the way, the group around the definition ofallowhashis useless, because you're changing no category code at definition time.
â egreg
Sep 2 at 11:27
@egreg aha! Removed the grouping (it was just remains from when I was experimenting).
â Andreas Storvik Strauman
Sep 2 at 12:02
add a comment |Â
up vote
8
down vote
favorite
up vote
8
down vote
favorite
If I make a command that takes one argument, and the argument should be allowed to take the unescaped default macro char: # as an argument and then treat it as a letter:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
The above works except it prints the following:One hash: ##, not two. Is this expected behaviour? How could one fix this?
macros catcodes
If I make a command that takes one argument, and the argument should be allowed to take the unescaped default macro char: # as an argument and then treat it as a letter:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
The above works except it prints the following:One hash: ##, not two. Is this expected behaviour? How could one fix this?
macros catcodes
macros catcodes
edited Sep 2 at 12:01
asked Sep 2 at 11:11
Andreas Storvik Strauman
2,257418
2,257418
1
When the macroallowhashis called, the category code of#is still 6, so it is doubled when the argument is absorbed. With a laterscantokensyou get two#characters with category code 11 (12 would be a better choice). By the way, the group around the definition ofallowhashis useless, because you're changing no category code at definition time.
â egreg
Sep 2 at 11:27
@egreg aha! Removed the grouping (it was just remains from when I was experimenting).
â Andreas Storvik Strauman
Sep 2 at 12:02
add a comment |Â
1
When the macroallowhashis called, the category code of#is still 6, so it is doubled when the argument is absorbed. With a laterscantokensyou get two#characters with category code 11 (12 would be a better choice). By the way, the group around the definition ofallowhashis useless, because you're changing no category code at definition time.
â egreg
Sep 2 at 11:27
@egreg aha! Removed the grouping (it was just remains from when I was experimenting).
â Andreas Storvik Strauman
Sep 2 at 12:02
1
1
When the macro
allowhash is called, the category code of # is still 6, so it is doubled when the argument is absorbed. With a later scantokens you get two # characters with category code 11 (12 would be a better choice). By the way, the group around the definition of allowhash is useless, because you're changing no category code at definition time.â egreg
Sep 2 at 11:27
When the macro
allowhash is called, the category code of # is still 6, so it is doubled when the argument is absorbed. With a later scantokens you get two # characters with category code 11 (12 would be a better choice). By the way, the group around the definition of allowhash is useless, because you're changing no category code at definition time.â egreg
Sep 2 at 11:27
@egreg aha! Removed the grouping (it was just remains from when I was experimenting).
â Andreas Storvik Strauman
Sep 2 at 12:02
@egreg aha! Removed the grouping (it was just remains from when I was experimenting).
â Andreas Storvik Strauman
Sep 2 at 12:02
add a comment |Â
3 Answers
3
active
oldest
votes
up vote
2
down vote
accepted
Let's look at your code:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
LaTeX does fetch an argument for allowhash, hereby tokenizing the tokens which form that argument under normal category-code-régime. Thus LaTeX does fetch an explicit-hash-character-token of category code 6(parameter) as a component of allowhash's argument.
When expanding allowhash, this token becomes a part of the content of toks@.
Thus toks@ contains an explicit-hash-character-token of category code 6(parameter).
Due to expandafter...the-trickery this explicit-hash-character-token of category code 6(parameter) ends up as a component of the ⟨general text⟩ of scantokens.
scantokens emulates unexpanded-writing the tokens from its ⟨general text⟩ into file and reading them back from the file and hereby tokenizing things under the current category-code-régime.
When writing to file or screen, explicit character tokens of category-code 6(parameter) get doubled.
Thus the hash gets doubled by scantokens unexpanded-writing-part.
Off the cuff I can only offer a routine ReplaceEveryHash which takes one argument and does replace each explicit catcode-6-character-token of the argument by its stringificationâthis mechanism does not act only on explicit catcode-6-hashes but on all explicit catcode-6-character-tokens.
documentclassarticle
makeatletter
%%=============================================================================
%% Paraphernalia:
%% UD@firstoftwo, UD@secondoftwo,
%% UD@PassFirstToSecond, UD@Exchange, UD@removespace
%% UD@CheckWhetherNull, UD@CheckWhetherBrace,
%% UD@CheckWhetherLeadingSpace, UD@ExtractFirstArg
%%=============================================================================
newcommandUD@firstoftwo[2]#1%
newcommandUD@secondoftwo[2]#2%
newcommandUD@PassFirstToSecond[2]#2#1%
newcommandUD@Exchange[2]#2#1%
newcommandUD@removespaceUD@firstoftwodefUD@removespace %
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% UD@CheckWhetherNull<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is empty>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is not empty>%
%%
%% The gist of this macro comes from Robert R. Schneck's ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
newcommandUD@CheckWhetherNull[1]%
romannumeral0expandafterUD@secondoftwostringexpandafter
UD@secondoftwoexpandafterexpandafterstring#1expandafter
UD@secondoftwostringexpandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@secondoftwoexpandafterexpandafterUD@firstoftwo UD@firstoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% UD@CheckWhetherBrace<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has leading
%% catcode-1-token>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has no leading
%% catcode-1-token>%
newcommandUD@CheckWhetherBrace[1]%
romannumeral0expandafterUD@secondoftwoexpandafterexpandafter%
string#1.expandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@firstoftwoexpandafterexpandafterUD@firstoftwo UD@secondoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether brace-balanced argument starts with a space-token
%%.............................................................................
%% UD@CheckWhetherLeadingSpace<Argument which is to be checked>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is a
%% space-token>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is not
%% a space-token>%
newcommandUD@CheckWhetherLeadingSpace[1]%
romannumeral0UD@CheckWhetherNull#1%
expandafterexpandafterUD@firstoftwo UD@secondoftwo%
expandafterUD@secondoftwostringUD@CheckWhetherLeadingSpaceB.#1 %
%
newcommandUD@CheckWhetherLeadingSpaceB%
longdefUD@CheckWhetherLeadingSpaceB#1 %
expandafterUD@CheckWhetherNullexpandafterUD@secondoftwo#1%
UD@ExchangeUD@firstoftwoUD@ExchangeUD@secondoftwo%
UD@Exchange expandafterexpandafterexpandafterexpandafter
expandafterexpandafterexpandafterexpandafterexpandafter
expandafterexpandafterUD@secondoftwoexpandafterstring%
%
%%-----------------------------------------------------------------------------
%% Extract first inner undelimited argument:
%%
%% UD@ExtractFirstArgABCDE yields A
%%
%% UD@ExtractFirstArgABCDE yields AB
%%.............................................................................
newcommandUD@RemoveTillUD@SelDOm%
longdefUD@RemoveTillUD@SelDOm#1#2UD@SelDOm#1%
newcommandUD@ExtractFirstArg[1]%
romannumeral0%
UD@ExtractFirstArgLoop#1UD@SelDOm%
%
newcommandUD@ExtractFirstArgLoop[1]%
expandafterUD@CheckWhetherNullexpandafterUD@firstoftwo#1%
#1%
expandafterUD@ExtractFirstArgLoopexpandafterUD@RemoveTillUD@SelDOm#1%
%
%%=============================================================================
%% ReplaceEveryHash<argument>%
%%
%% Each explicit catcode-6(parameter)-character-token of the <argument>
%% will be replaced by its stringification.
%%
%% You obtain the result after two expansion-steps, i.e.,
%% in expansion-contexts you get the result after "hitting"
%% ReplaceEveryHash by two expandafter.
%%
%% As a side-effect, the routine does replace matching pairs of explicit
%% character tokens of catcode 1 and 2 by matching pairs of curly braces
%% of catcode 1 and 2.
%% I suppose this won't be a problem in most situations as usually the
%% curly braces are the only characters of category code 1 / 2...
%%
%% This routine needs detokenize from the eTeX extensions.
%%-----------------------------------------------------------------------------
newcommandReplaceEveryHash[1]%
romannumeral0UD@ReplaceEveryHashLoop#1%
%
newcommandUD@ReplaceEveryHashLoop[2]%
UD@CheckWhetherNull#1 #2%
UD@CheckWhetherLeadingSpace#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@removespace#1#2 %
%
UD@CheckWhetherBrace#1%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafter%
expandafterUD@PassFirstToSecondexpandafter%
romannumeral0expandafterUD@ReplaceEveryHashLoop
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm%
#2%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#1%
%
expandafterUD@CheckWhetherHash
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm#1#2%
%
%
%
%
newcommandUD@CheckWhetherHash[3]%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterstring#1%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterdetokenize#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafterexpandafterUD@Exchange
expandafterstring#1#3%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2%
%
%
%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
%
%----------------------------------------------------------------------
newcommandallowhash[1]ReplaceEveryHash#1
newcommandallowanddetokenizehash[1]%
detokenizeexpandafterexpandafterexpandafterReplaceEveryHash#1%
%
makeatother
begindocument
allowhashOne hash: #, not two
begingroup
frenchspacing
ttfamily allowanddetokenizehashOne hash: #, not two. This time in braces:#####
endgroup
For comparison the effect of verb|detokenize| without prior hash-replacing:
begingroup
frenchspacing
ttfamily detokenizeOne hash: #, not two. This time in braces:#####
endgroup
enddocument

add a comment |Â
up vote
10
down vote

documentclassarticle
makeatletter
defallowhash%%%%
bgroupeveryeofegroupcatcode35=11relaxscantokens%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
or in a macro as requested in comments

documentclassarticle
makeatletter
defallowhash%%%%
bgroupcatcode35=11relaxafterassignmentegroupgdeffoo%
makeatother
begindocument
allowhashOne hash: #, not two
[foo] [foo]
enddocument
What if I wanted the result in a macro instead of just printing it?
â Andreas Storvik Strauman
Sep 2 at 11:35
@AndreasStorvikStrauman added a macro version
â David Carlisle
Sep 2 at 11:39
Could you add some words of explanation how the first macro works?
â siracusa
Sep 2 at 15:00
@siracusa it just makes#non-special without parsing the...as an argument ofallowhash, thescantokensisn't really needed at all except the end of file hook is used to insert an endgroup to restore the meaning of#
â David Carlisle
Sep 2 at 15:13
add a comment |Â
up vote
4
down vote
When scantokens acts, the # characters are already doubled, because they're absorbed as the argument to a macro.
With the l3regex module of expl3 it's easier:
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashm
tl_set:Nn l_tmpa_tl #1
regex_replace_all:nnN cP# cO# l_tmpa_tl
tl_use:N l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
enddocument

You can easily add support for saving the token list in a macro.
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashom
tl_set:Nn l_tmpa_tl #2
regex_replace_all:nnN cP# cO# l_tmpa_tl
IfNoValueTF #1
tl_use:N l_tmpa_tl
tl_set_eq:NN #1 l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
allowhash[foo]One hash: #, not two
textttmeaningfoo
enddocument

(There seems to be a double space in the picture, but it's only due to nonfrenchspacing.)
add a comment |Â
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
Let's look at your code:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
LaTeX does fetch an argument for allowhash, hereby tokenizing the tokens which form that argument under normal category-code-régime. Thus LaTeX does fetch an explicit-hash-character-token of category code 6(parameter) as a component of allowhash's argument.
When expanding allowhash, this token becomes a part of the content of toks@.
Thus toks@ contains an explicit-hash-character-token of category code 6(parameter).
Due to expandafter...the-trickery this explicit-hash-character-token of category code 6(parameter) ends up as a component of the ⟨general text⟩ of scantokens.
scantokens emulates unexpanded-writing the tokens from its ⟨general text⟩ into file and reading them back from the file and hereby tokenizing things under the current category-code-régime.
When writing to file or screen, explicit character tokens of category-code 6(parameter) get doubled.
Thus the hash gets doubled by scantokens unexpanded-writing-part.
Off the cuff I can only offer a routine ReplaceEveryHash which takes one argument and does replace each explicit catcode-6-character-token of the argument by its stringificationâthis mechanism does not act only on explicit catcode-6-hashes but on all explicit catcode-6-character-tokens.
documentclassarticle
makeatletter
%%=============================================================================
%% Paraphernalia:
%% UD@firstoftwo, UD@secondoftwo,
%% UD@PassFirstToSecond, UD@Exchange, UD@removespace
%% UD@CheckWhetherNull, UD@CheckWhetherBrace,
%% UD@CheckWhetherLeadingSpace, UD@ExtractFirstArg
%%=============================================================================
newcommandUD@firstoftwo[2]#1%
newcommandUD@secondoftwo[2]#2%
newcommandUD@PassFirstToSecond[2]#2#1%
newcommandUD@Exchange[2]#2#1%
newcommandUD@removespaceUD@firstoftwodefUD@removespace %
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% UD@CheckWhetherNull<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is empty>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is not empty>%
%%
%% The gist of this macro comes from Robert R. Schneck's ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
newcommandUD@CheckWhetherNull[1]%
romannumeral0expandafterUD@secondoftwostringexpandafter
UD@secondoftwoexpandafterexpandafterstring#1expandafter
UD@secondoftwostringexpandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@secondoftwoexpandafterexpandafterUD@firstoftwo UD@firstoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% UD@CheckWhetherBrace<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has leading
%% catcode-1-token>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has no leading
%% catcode-1-token>%
newcommandUD@CheckWhetherBrace[1]%
romannumeral0expandafterUD@secondoftwoexpandafterexpandafter%
string#1.expandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@firstoftwoexpandafterexpandafterUD@firstoftwo UD@secondoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether brace-balanced argument starts with a space-token
%%.............................................................................
%% UD@CheckWhetherLeadingSpace<Argument which is to be checked>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is a
%% space-token>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is not
%% a space-token>%
newcommandUD@CheckWhetherLeadingSpace[1]%
romannumeral0UD@CheckWhetherNull#1%
expandafterexpandafterUD@firstoftwo UD@secondoftwo%
expandafterUD@secondoftwostringUD@CheckWhetherLeadingSpaceB.#1 %
%
newcommandUD@CheckWhetherLeadingSpaceB%
longdefUD@CheckWhetherLeadingSpaceB#1 %
expandafterUD@CheckWhetherNullexpandafterUD@secondoftwo#1%
UD@ExchangeUD@firstoftwoUD@ExchangeUD@secondoftwo%
UD@Exchange expandafterexpandafterexpandafterexpandafter
expandafterexpandafterexpandafterexpandafterexpandafter
expandafterexpandafterUD@secondoftwoexpandafterstring%
%
%%-----------------------------------------------------------------------------
%% Extract first inner undelimited argument:
%%
%% UD@ExtractFirstArgABCDE yields A
%%
%% UD@ExtractFirstArgABCDE yields AB
%%.............................................................................
newcommandUD@RemoveTillUD@SelDOm%
longdefUD@RemoveTillUD@SelDOm#1#2UD@SelDOm#1%
newcommandUD@ExtractFirstArg[1]%
romannumeral0%
UD@ExtractFirstArgLoop#1UD@SelDOm%
%
newcommandUD@ExtractFirstArgLoop[1]%
expandafterUD@CheckWhetherNullexpandafterUD@firstoftwo#1%
#1%
expandafterUD@ExtractFirstArgLoopexpandafterUD@RemoveTillUD@SelDOm#1%
%
%%=============================================================================
%% ReplaceEveryHash<argument>%
%%
%% Each explicit catcode-6(parameter)-character-token of the <argument>
%% will be replaced by its stringification.
%%
%% You obtain the result after two expansion-steps, i.e.,
%% in expansion-contexts you get the result after "hitting"
%% ReplaceEveryHash by two expandafter.
%%
%% As a side-effect, the routine does replace matching pairs of explicit
%% character tokens of catcode 1 and 2 by matching pairs of curly braces
%% of catcode 1 and 2.
%% I suppose this won't be a problem in most situations as usually the
%% curly braces are the only characters of category code 1 / 2...
%%
%% This routine needs detokenize from the eTeX extensions.
%%-----------------------------------------------------------------------------
newcommandReplaceEveryHash[1]%
romannumeral0UD@ReplaceEveryHashLoop#1%
%
newcommandUD@ReplaceEveryHashLoop[2]%
UD@CheckWhetherNull#1 #2%
UD@CheckWhetherLeadingSpace#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@removespace#1#2 %
%
UD@CheckWhetherBrace#1%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafter%
expandafterUD@PassFirstToSecondexpandafter%
romannumeral0expandafterUD@ReplaceEveryHashLoop
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm%
#2%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#1%
%
expandafterUD@CheckWhetherHash
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm#1#2%
%
%
%
%
newcommandUD@CheckWhetherHash[3]%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterstring#1%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterdetokenize#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafterexpandafterUD@Exchange
expandafterstring#1#3%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2%
%
%
%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
%
%----------------------------------------------------------------------
newcommandallowhash[1]ReplaceEveryHash#1
newcommandallowanddetokenizehash[1]%
detokenizeexpandafterexpandafterexpandafterReplaceEveryHash#1%
%
makeatother
begindocument
allowhashOne hash: #, not two
begingroup
frenchspacing
ttfamily allowanddetokenizehashOne hash: #, not two. This time in braces:#####
endgroup
For comparison the effect of verb|detokenize| without prior hash-replacing:
begingroup
frenchspacing
ttfamily detokenizeOne hash: #, not two. This time in braces:#####
endgroup
enddocument

add a comment |Â
up vote
2
down vote
accepted
Let's look at your code:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
LaTeX does fetch an argument for allowhash, hereby tokenizing the tokens which form that argument under normal category-code-régime. Thus LaTeX does fetch an explicit-hash-character-token of category code 6(parameter) as a component of allowhash's argument.
When expanding allowhash, this token becomes a part of the content of toks@.
Thus toks@ contains an explicit-hash-character-token of category code 6(parameter).
Due to expandafter...the-trickery this explicit-hash-character-token of category code 6(parameter) ends up as a component of the ⟨general text⟩ of scantokens.
scantokens emulates unexpanded-writing the tokens from its ⟨general text⟩ into file and reading them back from the file and hereby tokenizing things under the current category-code-régime.
When writing to file or screen, explicit character tokens of category-code 6(parameter) get doubled.
Thus the hash gets doubled by scantokens unexpanded-writing-part.
Off the cuff I can only offer a routine ReplaceEveryHash which takes one argument and does replace each explicit catcode-6-character-token of the argument by its stringificationâthis mechanism does not act only on explicit catcode-6-hashes but on all explicit catcode-6-character-tokens.
documentclassarticle
makeatletter
%%=============================================================================
%% Paraphernalia:
%% UD@firstoftwo, UD@secondoftwo,
%% UD@PassFirstToSecond, UD@Exchange, UD@removespace
%% UD@CheckWhetherNull, UD@CheckWhetherBrace,
%% UD@CheckWhetherLeadingSpace, UD@ExtractFirstArg
%%=============================================================================
newcommandUD@firstoftwo[2]#1%
newcommandUD@secondoftwo[2]#2%
newcommandUD@PassFirstToSecond[2]#2#1%
newcommandUD@Exchange[2]#2#1%
newcommandUD@removespaceUD@firstoftwodefUD@removespace %
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% UD@CheckWhetherNull<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is empty>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is not empty>%
%%
%% The gist of this macro comes from Robert R. Schneck's ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
newcommandUD@CheckWhetherNull[1]%
romannumeral0expandafterUD@secondoftwostringexpandafter
UD@secondoftwoexpandafterexpandafterstring#1expandafter
UD@secondoftwostringexpandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@secondoftwoexpandafterexpandafterUD@firstoftwo UD@firstoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% UD@CheckWhetherBrace<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has leading
%% catcode-1-token>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has no leading
%% catcode-1-token>%
newcommandUD@CheckWhetherBrace[1]%
romannumeral0expandafterUD@secondoftwoexpandafterexpandafter%
string#1.expandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@firstoftwoexpandafterexpandafterUD@firstoftwo UD@secondoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether brace-balanced argument starts with a space-token
%%.............................................................................
%% UD@CheckWhetherLeadingSpace<Argument which is to be checked>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is a
%% space-token>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is not
%% a space-token>%
newcommandUD@CheckWhetherLeadingSpace[1]%
romannumeral0UD@CheckWhetherNull#1%
expandafterexpandafterUD@firstoftwo UD@secondoftwo%
expandafterUD@secondoftwostringUD@CheckWhetherLeadingSpaceB.#1 %
%
newcommandUD@CheckWhetherLeadingSpaceB%
longdefUD@CheckWhetherLeadingSpaceB#1 %
expandafterUD@CheckWhetherNullexpandafterUD@secondoftwo#1%
UD@ExchangeUD@firstoftwoUD@ExchangeUD@secondoftwo%
UD@Exchange expandafterexpandafterexpandafterexpandafter
expandafterexpandafterexpandafterexpandafterexpandafter
expandafterexpandafterUD@secondoftwoexpandafterstring%
%
%%-----------------------------------------------------------------------------
%% Extract first inner undelimited argument:
%%
%% UD@ExtractFirstArgABCDE yields A
%%
%% UD@ExtractFirstArgABCDE yields AB
%%.............................................................................
newcommandUD@RemoveTillUD@SelDOm%
longdefUD@RemoveTillUD@SelDOm#1#2UD@SelDOm#1%
newcommandUD@ExtractFirstArg[1]%
romannumeral0%
UD@ExtractFirstArgLoop#1UD@SelDOm%
%
newcommandUD@ExtractFirstArgLoop[1]%
expandafterUD@CheckWhetherNullexpandafterUD@firstoftwo#1%
#1%
expandafterUD@ExtractFirstArgLoopexpandafterUD@RemoveTillUD@SelDOm#1%
%
%%=============================================================================
%% ReplaceEveryHash<argument>%
%%
%% Each explicit catcode-6(parameter)-character-token of the <argument>
%% will be replaced by its stringification.
%%
%% You obtain the result after two expansion-steps, i.e.,
%% in expansion-contexts you get the result after "hitting"
%% ReplaceEveryHash by two expandafter.
%%
%% As a side-effect, the routine does replace matching pairs of explicit
%% character tokens of catcode 1 and 2 by matching pairs of curly braces
%% of catcode 1 and 2.
%% I suppose this won't be a problem in most situations as usually the
%% curly braces are the only characters of category code 1 / 2...
%%
%% This routine needs detokenize from the eTeX extensions.
%%-----------------------------------------------------------------------------
newcommandReplaceEveryHash[1]%
romannumeral0UD@ReplaceEveryHashLoop#1%
%
newcommandUD@ReplaceEveryHashLoop[2]%
UD@CheckWhetherNull#1 #2%
UD@CheckWhetherLeadingSpace#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@removespace#1#2 %
%
UD@CheckWhetherBrace#1%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafter%
expandafterUD@PassFirstToSecondexpandafter%
romannumeral0expandafterUD@ReplaceEveryHashLoop
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm%
#2%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#1%
%
expandafterUD@CheckWhetherHash
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm#1#2%
%
%
%
%
newcommandUD@CheckWhetherHash[3]%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterstring#1%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterdetokenize#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafterexpandafterUD@Exchange
expandafterstring#1#3%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2%
%
%
%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
%
%----------------------------------------------------------------------
newcommandallowhash[1]ReplaceEveryHash#1
newcommandallowanddetokenizehash[1]%
detokenizeexpandafterexpandafterexpandafterReplaceEveryHash#1%
%
makeatother
begindocument
allowhashOne hash: #, not two
begingroup
frenchspacing
ttfamily allowanddetokenizehashOne hash: #, not two. This time in braces:#####
endgroup
For comparison the effect of verb|detokenize| without prior hash-replacing:
begingroup
frenchspacing
ttfamily detokenizeOne hash: #, not two. This time in braces:#####
endgroup
enddocument

add a comment |Â
up vote
2
down vote
accepted
up vote
2
down vote
accepted
Let's look at your code:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
LaTeX does fetch an argument for allowhash, hereby tokenizing the tokens which form that argument under normal category-code-régime. Thus LaTeX does fetch an explicit-hash-character-token of category code 6(parameter) as a component of allowhash's argument.
When expanding allowhash, this token becomes a part of the content of toks@.
Thus toks@ contains an explicit-hash-character-token of category code 6(parameter).
Due to expandafter...the-trickery this explicit-hash-character-token of category code 6(parameter) ends up as a component of the ⟨general text⟩ of scantokens.
scantokens emulates unexpanded-writing the tokens from its ⟨general text⟩ into file and reading them back from the file and hereby tokenizing things under the current category-code-régime.
When writing to file or screen, explicit character tokens of category-code 6(parameter) get doubled.
Thus the hash gets doubled by scantokens unexpanded-writing-part.
Off the cuff I can only offer a routine ReplaceEveryHash which takes one argument and does replace each explicit catcode-6-character-token of the argument by its stringificationâthis mechanism does not act only on explicit catcode-6-hashes but on all explicit catcode-6-character-tokens.
documentclassarticle
makeatletter
%%=============================================================================
%% Paraphernalia:
%% UD@firstoftwo, UD@secondoftwo,
%% UD@PassFirstToSecond, UD@Exchange, UD@removespace
%% UD@CheckWhetherNull, UD@CheckWhetherBrace,
%% UD@CheckWhetherLeadingSpace, UD@ExtractFirstArg
%%=============================================================================
newcommandUD@firstoftwo[2]#1%
newcommandUD@secondoftwo[2]#2%
newcommandUD@PassFirstToSecond[2]#2#1%
newcommandUD@Exchange[2]#2#1%
newcommandUD@removespaceUD@firstoftwodefUD@removespace %
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% UD@CheckWhetherNull<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is empty>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is not empty>%
%%
%% The gist of this macro comes from Robert R. Schneck's ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
newcommandUD@CheckWhetherNull[1]%
romannumeral0expandafterUD@secondoftwostringexpandafter
UD@secondoftwoexpandafterexpandafterstring#1expandafter
UD@secondoftwostringexpandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@secondoftwoexpandafterexpandafterUD@firstoftwo UD@firstoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% UD@CheckWhetherBrace<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has leading
%% catcode-1-token>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has no leading
%% catcode-1-token>%
newcommandUD@CheckWhetherBrace[1]%
romannumeral0expandafterUD@secondoftwoexpandafterexpandafter%
string#1.expandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@firstoftwoexpandafterexpandafterUD@firstoftwo UD@secondoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether brace-balanced argument starts with a space-token
%%.............................................................................
%% UD@CheckWhetherLeadingSpace<Argument which is to be checked>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is a
%% space-token>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is not
%% a space-token>%
newcommandUD@CheckWhetherLeadingSpace[1]%
romannumeral0UD@CheckWhetherNull#1%
expandafterexpandafterUD@firstoftwo UD@secondoftwo%
expandafterUD@secondoftwostringUD@CheckWhetherLeadingSpaceB.#1 %
%
newcommandUD@CheckWhetherLeadingSpaceB%
longdefUD@CheckWhetherLeadingSpaceB#1 %
expandafterUD@CheckWhetherNullexpandafterUD@secondoftwo#1%
UD@ExchangeUD@firstoftwoUD@ExchangeUD@secondoftwo%
UD@Exchange expandafterexpandafterexpandafterexpandafter
expandafterexpandafterexpandafterexpandafterexpandafter
expandafterexpandafterUD@secondoftwoexpandafterstring%
%
%%-----------------------------------------------------------------------------
%% Extract first inner undelimited argument:
%%
%% UD@ExtractFirstArgABCDE yields A
%%
%% UD@ExtractFirstArgABCDE yields AB
%%.............................................................................
newcommandUD@RemoveTillUD@SelDOm%
longdefUD@RemoveTillUD@SelDOm#1#2UD@SelDOm#1%
newcommandUD@ExtractFirstArg[1]%
romannumeral0%
UD@ExtractFirstArgLoop#1UD@SelDOm%
%
newcommandUD@ExtractFirstArgLoop[1]%
expandafterUD@CheckWhetherNullexpandafterUD@firstoftwo#1%
#1%
expandafterUD@ExtractFirstArgLoopexpandafterUD@RemoveTillUD@SelDOm#1%
%
%%=============================================================================
%% ReplaceEveryHash<argument>%
%%
%% Each explicit catcode-6(parameter)-character-token of the <argument>
%% will be replaced by its stringification.
%%
%% You obtain the result after two expansion-steps, i.e.,
%% in expansion-contexts you get the result after "hitting"
%% ReplaceEveryHash by two expandafter.
%%
%% As a side-effect, the routine does replace matching pairs of explicit
%% character tokens of catcode 1 and 2 by matching pairs of curly braces
%% of catcode 1 and 2.
%% I suppose this won't be a problem in most situations as usually the
%% curly braces are the only characters of category code 1 / 2...
%%
%% This routine needs detokenize from the eTeX extensions.
%%-----------------------------------------------------------------------------
newcommandReplaceEveryHash[1]%
romannumeral0UD@ReplaceEveryHashLoop#1%
%
newcommandUD@ReplaceEveryHashLoop[2]%
UD@CheckWhetherNull#1 #2%
UD@CheckWhetherLeadingSpace#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@removespace#1#2 %
%
UD@CheckWhetherBrace#1%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafter%
expandafterUD@PassFirstToSecondexpandafter%
romannumeral0expandafterUD@ReplaceEveryHashLoop
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm%
#2%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#1%
%
expandafterUD@CheckWhetherHash
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm#1#2%
%
%
%
%
newcommandUD@CheckWhetherHash[3]%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterstring#1%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterdetokenize#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafterexpandafterUD@Exchange
expandafterstring#1#3%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2%
%
%
%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
%
%----------------------------------------------------------------------
newcommandallowhash[1]ReplaceEveryHash#1
newcommandallowanddetokenizehash[1]%
detokenizeexpandafterexpandafterexpandafterReplaceEveryHash#1%
%
makeatother
begindocument
allowhashOne hash: #, not two
begingroup
frenchspacing
ttfamily allowanddetokenizehashOne hash: #, not two. This time in braces:#####
endgroup
For comparison the effect of verb|detokenize| without prior hash-replacing:
begingroup
frenchspacing
ttfamily detokenizeOne hash: #, not two. This time in braces:#####
endgroup
enddocument

Let's look at your code:
documentclassarticle
makeatletter
defallowhash#1
toks@#1catcode`#11relaxscantokensexpandafterthetoks@%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
LaTeX does fetch an argument for allowhash, hereby tokenizing the tokens which form that argument under normal category-code-régime. Thus LaTeX does fetch an explicit-hash-character-token of category code 6(parameter) as a component of allowhash's argument.
When expanding allowhash, this token becomes a part of the content of toks@.
Thus toks@ contains an explicit-hash-character-token of category code 6(parameter).
Due to expandafter...the-trickery this explicit-hash-character-token of category code 6(parameter) ends up as a component of the ⟨general text⟩ of scantokens.
scantokens emulates unexpanded-writing the tokens from its ⟨general text⟩ into file and reading them back from the file and hereby tokenizing things under the current category-code-régime.
When writing to file or screen, explicit character tokens of category-code 6(parameter) get doubled.
Thus the hash gets doubled by scantokens unexpanded-writing-part.
Off the cuff I can only offer a routine ReplaceEveryHash which takes one argument and does replace each explicit catcode-6-character-token of the argument by its stringificationâthis mechanism does not act only on explicit catcode-6-hashes but on all explicit catcode-6-character-tokens.
documentclassarticle
makeatletter
%%=============================================================================
%% Paraphernalia:
%% UD@firstoftwo, UD@secondoftwo,
%% UD@PassFirstToSecond, UD@Exchange, UD@removespace
%% UD@CheckWhetherNull, UD@CheckWhetherBrace,
%% UD@CheckWhetherLeadingSpace, UD@ExtractFirstArg
%%=============================================================================
newcommandUD@firstoftwo[2]#1%
newcommandUD@secondoftwo[2]#2%
newcommandUD@PassFirstToSecond[2]#2#1%
newcommandUD@Exchange[2]#2#1%
newcommandUD@removespaceUD@firstoftwodefUD@removespace %
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% UD@CheckWhetherNull<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is empty>%
%% <Tokens to be delivered in case that argument
%% which is to be checked is not empty>%
%%
%% The gist of this macro comes from Robert R. Schneck's ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
newcommandUD@CheckWhetherNull[1]%
romannumeral0expandafterUD@secondoftwostringexpandafter
UD@secondoftwoexpandafterexpandafterstring#1expandafter
UD@secondoftwostringexpandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@secondoftwoexpandafterexpandafterUD@firstoftwo UD@firstoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether argument's first token is a catcode-1-character
%%.............................................................................
%% UD@CheckWhetherBrace<Argument which is to be checked>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has leading
%% catcode-1-token>%
%% <Tokens to be delivered in case that argument
%% which is to be checked has no leading
%% catcode-1-token>%
newcommandUD@CheckWhetherBrace[1]%
romannumeral0expandafterUD@secondoftwoexpandafterexpandafter%
string#1.expandafterUD@firstoftwoexpandafterexpandafter
UD@secondoftwostringexpandafterexpandafterUD@firstoftwo %
UD@firstoftwoexpandafterexpandafterUD@firstoftwo UD@secondoftwo%
%
%%-----------------------------------------------------------------------------
%% Check whether brace-balanced argument starts with a space-token
%%.............................................................................
%% UD@CheckWhetherLeadingSpace<Argument which is to be checked>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is a
%% space-token>%
%% <Tokens to be delivered in case <argument
%% which is to be checked>'s 1st token is not
%% a space-token>%
newcommandUD@CheckWhetherLeadingSpace[1]%
romannumeral0UD@CheckWhetherNull#1%
expandafterexpandafterUD@firstoftwo UD@secondoftwo%
expandafterUD@secondoftwostringUD@CheckWhetherLeadingSpaceB.#1 %
%
newcommandUD@CheckWhetherLeadingSpaceB%
longdefUD@CheckWhetherLeadingSpaceB#1 %
expandafterUD@CheckWhetherNullexpandafterUD@secondoftwo#1%
UD@ExchangeUD@firstoftwoUD@ExchangeUD@secondoftwo%
UD@Exchange expandafterexpandafterexpandafterexpandafter
expandafterexpandafterexpandafterexpandafterexpandafter
expandafterexpandafterUD@secondoftwoexpandafterstring%
%
%%-----------------------------------------------------------------------------
%% Extract first inner undelimited argument:
%%
%% UD@ExtractFirstArgABCDE yields A
%%
%% UD@ExtractFirstArgABCDE yields AB
%%.............................................................................
newcommandUD@RemoveTillUD@SelDOm%
longdefUD@RemoveTillUD@SelDOm#1#2UD@SelDOm#1%
newcommandUD@ExtractFirstArg[1]%
romannumeral0%
UD@ExtractFirstArgLoop#1UD@SelDOm%
%
newcommandUD@ExtractFirstArgLoop[1]%
expandafterUD@CheckWhetherNullexpandafterUD@firstoftwo#1%
#1%
expandafterUD@ExtractFirstArgLoopexpandafterUD@RemoveTillUD@SelDOm#1%
%
%%=============================================================================
%% ReplaceEveryHash<argument>%
%%
%% Each explicit catcode-6(parameter)-character-token of the <argument>
%% will be replaced by its stringification.
%%
%% You obtain the result after two expansion-steps, i.e.,
%% in expansion-contexts you get the result after "hitting"
%% ReplaceEveryHash by two expandafter.
%%
%% As a side-effect, the routine does replace matching pairs of explicit
%% character tokens of catcode 1 and 2 by matching pairs of curly braces
%% of catcode 1 and 2.
%% I suppose this won't be a problem in most situations as usually the
%% curly braces are the only characters of category code 1 / 2...
%%
%% This routine needs detokenize from the eTeX extensions.
%%-----------------------------------------------------------------------------
newcommandReplaceEveryHash[1]%
romannumeral0UD@ReplaceEveryHashLoop#1%
%
newcommandUD@ReplaceEveryHashLoop[2]%
UD@CheckWhetherNull#1 #2%
UD@CheckWhetherLeadingSpace#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@removespace#1#2 %
%
UD@CheckWhetherBrace#1%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafter%
expandafterUD@PassFirstToSecondexpandafter%
romannumeral0expandafterUD@ReplaceEveryHashLoop
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm%
#2%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#1%
%
expandafterUD@CheckWhetherHash
romannumeral0UD@ExtractFirstArgLoop#1UD@SelDOm#1#2%
%
%
%
%
newcommandUD@CheckWhetherHash[3]%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterstring#1%
expandafterexpandafterexpandafterUD@CheckWhetherNull
expandafterexpandafterexpandafter%
expandafterUD@firstoftwo
expandafterexpandafterdetokenize#1%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
expandafterexpandafterexpandafterUD@PassFirstToSecond
expandafterexpandafterexpandafterexpandafterUD@Exchange
expandafterstring#1#3%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2%
%
%
%
expandafterUD@ReplaceEveryHashLoop
expandafterUD@firstoftwo#2#3#1%
%
%
%----------------------------------------------------------------------
newcommandallowhash[1]ReplaceEveryHash#1
newcommandallowanddetokenizehash[1]%
detokenizeexpandafterexpandafterexpandafterReplaceEveryHash#1%
%
makeatother
begindocument
allowhashOne hash: #, not two
begingroup
frenchspacing
ttfamily allowanddetokenizehashOne hash: #, not two. This time in braces:#####
endgroup
For comparison the effect of verb|detokenize| without prior hash-replacing:
begingroup
frenchspacing
ttfamily detokenizeOne hash: #, not two. This time in braces:#####
endgroup
enddocument

edited Sep 3 at 11:11
Phelype Oleinik
16.4k33567
16.4k33567
answered Sep 2 at 21:29
Ulrich Diez
3,350414
3,350414
add a comment |Â
add a comment |Â
up vote
10
down vote

documentclassarticle
makeatletter
defallowhash%%%%
bgroupeveryeofegroupcatcode35=11relaxscantokens%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
or in a macro as requested in comments

documentclassarticle
makeatletter
defallowhash%%%%
bgroupcatcode35=11relaxafterassignmentegroupgdeffoo%
makeatother
begindocument
allowhashOne hash: #, not two
[foo] [foo]
enddocument
What if I wanted the result in a macro instead of just printing it?
â Andreas Storvik Strauman
Sep 2 at 11:35
@AndreasStorvikStrauman added a macro version
â David Carlisle
Sep 2 at 11:39
Could you add some words of explanation how the first macro works?
â siracusa
Sep 2 at 15:00
@siracusa it just makes#non-special without parsing the...as an argument ofallowhash, thescantokensisn't really needed at all except the end of file hook is used to insert an endgroup to restore the meaning of#
â David Carlisle
Sep 2 at 15:13
add a comment |Â
up vote
10
down vote

documentclassarticle
makeatletter
defallowhash%%%%
bgroupeveryeofegroupcatcode35=11relaxscantokens%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
or in a macro as requested in comments

documentclassarticle
makeatletter
defallowhash%%%%
bgroupcatcode35=11relaxafterassignmentegroupgdeffoo%
makeatother
begindocument
allowhashOne hash: #, not two
[foo] [foo]
enddocument
What if I wanted the result in a macro instead of just printing it?
â Andreas Storvik Strauman
Sep 2 at 11:35
@AndreasStorvikStrauman added a macro version
â David Carlisle
Sep 2 at 11:39
Could you add some words of explanation how the first macro works?
â siracusa
Sep 2 at 15:00
@siracusa it just makes#non-special without parsing the...as an argument ofallowhash, thescantokensisn't really needed at all except the end of file hook is used to insert an endgroup to restore the meaning of#
â David Carlisle
Sep 2 at 15:13
add a comment |Â
up vote
10
down vote
up vote
10
down vote

documentclassarticle
makeatletter
defallowhash%%%%
bgroupeveryeofegroupcatcode35=11relaxscantokens%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
or in a macro as requested in comments

documentclassarticle
makeatletter
defallowhash%%%%
bgroupcatcode35=11relaxafterassignmentegroupgdeffoo%
makeatother
begindocument
allowhashOne hash: #, not two
[foo] [foo]
enddocument

documentclassarticle
makeatletter
defallowhash%%%%
bgroupeveryeofegroupcatcode35=11relaxscantokens%
makeatother
begindocument
allowhashOne hash: #, not two
enddocument
or in a macro as requested in comments

documentclassarticle
makeatletter
defallowhash%%%%
bgroupcatcode35=11relaxafterassignmentegroupgdeffoo%
makeatother
begindocument
allowhashOne hash: #, not two
[foo] [foo]
enddocument
edited Sep 2 at 11:39
answered Sep 2 at 11:24
David Carlisle
467k3810951818
467k3810951818
What if I wanted the result in a macro instead of just printing it?
â Andreas Storvik Strauman
Sep 2 at 11:35
@AndreasStorvikStrauman added a macro version
â David Carlisle
Sep 2 at 11:39
Could you add some words of explanation how the first macro works?
â siracusa
Sep 2 at 15:00
@siracusa it just makes#non-special without parsing the...as an argument ofallowhash, thescantokensisn't really needed at all except the end of file hook is used to insert an endgroup to restore the meaning of#
â David Carlisle
Sep 2 at 15:13
add a comment |Â
What if I wanted the result in a macro instead of just printing it?
â Andreas Storvik Strauman
Sep 2 at 11:35
@AndreasStorvikStrauman added a macro version
â David Carlisle
Sep 2 at 11:39
Could you add some words of explanation how the first macro works?
â siracusa
Sep 2 at 15:00
@siracusa it just makes#non-special without parsing the...as an argument ofallowhash, thescantokensisn't really needed at all except the end of file hook is used to insert an endgroup to restore the meaning of#
â David Carlisle
Sep 2 at 15:13
What if I wanted the result in a macro instead of just printing it?
â Andreas Storvik Strauman
Sep 2 at 11:35
What if I wanted the result in a macro instead of just printing it?
â Andreas Storvik Strauman
Sep 2 at 11:35
@AndreasStorvikStrauman added a macro version
â David Carlisle
Sep 2 at 11:39
@AndreasStorvikStrauman added a macro version
â David Carlisle
Sep 2 at 11:39
Could you add some words of explanation how the first macro works?
â siracusa
Sep 2 at 15:00
Could you add some words of explanation how the first macro works?
â siracusa
Sep 2 at 15:00
@siracusa it just makes
# non-special without parsing the ... as an argument of allowhash, the scantokens isn't really needed at all except the end of file hook is used to insert an endgroup to restore the meaning of #â David Carlisle
Sep 2 at 15:13
@siracusa it just makes
# non-special without parsing the ... as an argument of allowhash, the scantokens isn't really needed at all except the end of file hook is used to insert an endgroup to restore the meaning of #â David Carlisle
Sep 2 at 15:13
add a comment |Â
up vote
4
down vote
When scantokens acts, the # characters are already doubled, because they're absorbed as the argument to a macro.
With the l3regex module of expl3 it's easier:
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashm
tl_set:Nn l_tmpa_tl #1
regex_replace_all:nnN cP# cO# l_tmpa_tl
tl_use:N l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
enddocument

You can easily add support for saving the token list in a macro.
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashom
tl_set:Nn l_tmpa_tl #2
regex_replace_all:nnN cP# cO# l_tmpa_tl
IfNoValueTF #1
tl_use:N l_tmpa_tl
tl_set_eq:NN #1 l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
allowhash[foo]One hash: #, not two
textttmeaningfoo
enddocument

(There seems to be a double space in the picture, but it's only due to nonfrenchspacing.)
add a comment |Â
up vote
4
down vote
When scantokens acts, the # characters are already doubled, because they're absorbed as the argument to a macro.
With the l3regex module of expl3 it's easier:
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashm
tl_set:Nn l_tmpa_tl #1
regex_replace_all:nnN cP# cO# l_tmpa_tl
tl_use:N l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
enddocument

You can easily add support for saving the token list in a macro.
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashom
tl_set:Nn l_tmpa_tl #2
regex_replace_all:nnN cP# cO# l_tmpa_tl
IfNoValueTF #1
tl_use:N l_tmpa_tl
tl_set_eq:NN #1 l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
allowhash[foo]One hash: #, not two
textttmeaningfoo
enddocument

(There seems to be a double space in the picture, but it's only due to nonfrenchspacing.)
add a comment |Â
up vote
4
down vote
up vote
4
down vote
When scantokens acts, the # characters are already doubled, because they're absorbed as the argument to a macro.
With the l3regex module of expl3 it's easier:
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashm
tl_set:Nn l_tmpa_tl #1
regex_replace_all:nnN cP# cO# l_tmpa_tl
tl_use:N l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
enddocument

You can easily add support for saving the token list in a macro.
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashom
tl_set:Nn l_tmpa_tl #2
regex_replace_all:nnN cP# cO# l_tmpa_tl
IfNoValueTF #1
tl_use:N l_tmpa_tl
tl_set_eq:NN #1 l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
allowhash[foo]One hash: #, not two
textttmeaningfoo
enddocument

(There seems to be a double space in the picture, but it's only due to nonfrenchspacing.)
When scantokens acts, the # characters are already doubled, because they're absorbed as the argument to a macro.
With the l3regex module of expl3 it's easier:
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashm
tl_set:Nn l_tmpa_tl #1
regex_replace_all:nnN cP# cO# l_tmpa_tl
tl_use:N l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
enddocument

You can easily add support for saving the token list in a macro.
documentclassarticle
usepackagexparse
ExplSyntaxOn
NewDocumentCommandallowhashom
tl_set:Nn l_tmpa_tl #2
regex_replace_all:nnN cP# cO# l_tmpa_tl
IfNoValueTF #1
tl_use:N l_tmpa_tl
tl_set_eq:NN #1 l_tmpa_tl
ExplSyntaxOff
begindocument
allowhashOne hash: #, not two
allowhash[foo]One hash: #, not two
textttmeaningfoo
enddocument

(There seems to be a double space in the picture, but it's only due to nonfrenchspacing.)
edited Sep 2 at 11:46
answered Sep 2 at 11:41
egreg
682k8318133061
682k8318133061
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f448972%2fmacro-char-doubles-when-made-letter%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
When the macro
allowhashis called, the category code of#is still 6, so it is doubled when the argument is absorbed. With a laterscantokensyou get two#characters with category code 11 (12 would be a better choice). By the way, the group around the definition ofallowhashis useless, because you're changing no category code at definition time.â egreg
Sep 2 at 11:27
@egreg aha! Removed the grouping (it was just remains from when I was experimenting).
â Andreas Storvik Strauman
Sep 2 at 12:02