String split iterator

Code:

std::vector<std::string> StringSplit(const std::string &text, int separator) {
        std::vector<std::string> vs(text.empty() ? 0 : 1);
        for (const char ch : text) {
                if (ch == separator) {
                        vs.emplace_back();
                } else {
                        vs.back() += ch;
                }
        }
        return vs;
}

People like to say that assembly is cryptic and hard to understand. I know what the above C++ does, but there is little in the syntax that explains the function - the range loop and high-level objects hide what is happening. How much object code do you think is generated by the compiler for that obtuse flexibility? (Plus several support functions!)

What about the common use case? I mean, we aren't doing bioinformatics with this code - there are special parallel SIMD-type algorithms for that. We have few parameters in a config file, or on the command line -- that's the work being done in 99% of cases.

The code for those use cases is very tiny. We don't want to be copying data around unless it really needs to persist. Like a file name - it's probably being converted into a full path name string. Like flags and options - they will configure internal state and the string is not needed afterward. In that regard we arrive here:

Code:

SKIP_EMPTIES := 1 ; consume multiple splits
macro ITERATOR__STRING_SPLIT char:' ',FUNCTION&
; RDI:  string to scan
; RCX:  length of string in characters, >0
; AX:   character to split on

        local scan,last,skip,done,last_clear
scan:
        mov al,char
        mov rsi,rdi
        repnz scasb
        push rdi
        jnz last
        sub edi,1               ; don't count the split character in length
last:
        push rcx
        sub edi,esi             ; length without terminator
        if definite SKIP_EMPTIES and SKIP_EMPTIES
                jz skip
        end if

        FUNCTION ; (address:RSI, length:RDI)
skip:
        pop rcx                 ; characters to go
        pop rdi                 ; start
        jrcxz done
        jmp scan
done:
        if definite SKIP_EMPTIES and SKIP_EMPTIES
        else ; check for last empty
                cmp byte [rdi-1],char
                jnz last_clear
                mov edi,ecx
                FUNCTION ; (address:RSI, length:RDI)
        last_clear:
        end if
end macro

With a few bytes we have much greater flexibility. The test cases used to verify the iterator:

Code:

        iterate test_case,\
                "1",\           ; single value
                "|",\           ; two empty values
                "||",\          ; three empty values
                "1|2",\         ; two values
                "|1|2",\        ; empty, middle, last
                "1|2|"          ; first, middle, empty
                ; empty string not allowed

                WriteConsoleA [.hCon],ADDR .LF,1,0,0
                mov rax,test_case
                push rax
                lea rdi,[rsp]
                mov ecx,lengthof test_case
                ITERATOR__STRING_SPLIT '|',call PrintCase
                pop rax
        end iterate

There is no guess-work. Change a couple bytes to process other data types. There is a use-case we don't want to blindly use this iterator on: the one where the separator can be from a set of values (like spaces and tabs). It isn't terribly difficult to add the feature though. Enjoy.