Cannot execute command of exactly 64 length

[marc7s]

Hi,

I've been using the library but noticed a weird bug. I was trying to get the MD5 hash of a file, but noticed that if the length of the command was specifically 64, then it would crash.

Normally, I would get the echoed command back (I tried disabling this, but seems the Telnet server I am using does not listen to those IAC commands for disabling echo), then \r\n, then the output of the command, then \r\n again and finally the input prompt (user@...). But if the command is exactly 64 characters long, I get just a \r instead and some x in there. But I don't get the output. My fix now is just checking if the command is 64 characters long, and if so prepend a space (instead of running cmd, run cmd, making it 65 characters long.

My code looks something like this:

reader, writer = await telnetlib3.open_connection("put your IP here", encoding="utf8", shell=None)

sep = "user@..., put the prompt here"

working_cmd = "echo something"
writer.write(working_cmd + "\n")
working_output = await reader.readuntil(separator=sep)
print(working_output.decode()) # works


non_working_cmd = "echo ".ljust(64, "a") # total length: 64 chars
writer.write(non_working_cmd + "\n")
non_working_output = await reader.readuntil(separator=sep)
print(non_working_output.decode()) # does not work

[jquast]

I would suggest the "window size" of the terminal is 80, and the prompt is probably about 16 characters long, and this unwanted effect may be due to the command input reaching the last column, and whatever effect that may cause, like redrawing the prompt and scrolling horizontally

I get just a \r instead

This is exactly what is sent to move the cursor back to column 0, probably to redraw the prompt, and, your input might now be clipped

[marc7s]

@jquast
That was my theory as well, but how do I fix it? Is it the telnet server causing the issue or something in the library? Can the terminal window size be changed?

Also, I tried with multiples of 64 and the only issue was with 64 specifically. So then the window size theory does not seem plausible anymore? My fix specifically checks if the command length is 64 chars, and if so pads a space to make it 65. But I also tried with commands up to 200 characters long, and they all still worked even though the padding only affects the 64 case. If the window size was the issue, shouldn't a command of length 128 and 192 also break?

[jquast]

Oh, new idea, try this, change:

reader, writer = await telnetlib3.open_connection("put your IP here", encoding="utf8", shell=None)

to

reader, writer = await telnetlib3.open_connection("put your IP here", encoding="utf8", shell=None, limit=99999)

and see if that changes anything,

[jquast]

My other recommendation is to call the match function twice,

non_working_output = await reader.readuntil(separator=sep)
print(non_working_output.decode()) # does not work
non_working_output = await reader.readuntil(separator=sep)
print(non_working_output.decode()) # do it again

and see if there is any additional output that helps

[marc7s]

reader, writer = await telnetlib3.open_connection("put your IP here", encoding="utf8", shell=None)

to

reader, writer = await telnetlib3.open_connection("put your IP here", encoding="utf8", shell=None, limit=99999)

and see if that changes anything,

Does not work, unfortunately.

[marc7s]

My other recommendation is to call the match function twice,

non_working_output = await reader.readuntil(separator=sep)
print(non_working_output.decode()) # does not work
non_working_output = await reader.readuntil(separator=sep)
print(non_working_output.decode()) # do it again

and see if there is any additional output that helps

There is, which I found during my prior testing. But it is no better than prepending a space to the command, since this only works in the specific case where the command is 64 in length. In all other cases, it gets stuck waiting for the separator which never comes.

In all other cases, running the echo aaa... command would give me b'echo aaaaaaaa\r\naaaaaaaa\r\nuser@...' back.
But in the case where it is 64 in length, I instead get (of course shortened, but imagine 64 a's) b'echo aaaaaaaa \r\x1b[Auser@...'.
So an additional space after the last a, and then the sequence [27, 91, 65] in decimal ASCII. The first of which is the interesting one, an escape character.

But my question still remains: is this an issue with my specific telnet server? Or is it caused by the telnetlib3 library? Are you able to replicate this issue on your end, with a different telnet server? And also, does the terminal size theory hold up with my comment #103 (comment) in mind?

[jquast]

But my question still remains: is this an issue with my specific telnet server? Or is it caused by the telnetlib3 library? Are you able to replicate this issue on your end, with a different telnet server?

This library contains a server as well as a client. I wrote ~600LOC server + client examples (ai assisted, a bit ugly sorry), as an automatic test to try to reproduce your problem, and I could not reproduce it, I made many variations, if the problem is with telnetlib3 it is very well hidden.

If you can provide a failing test case that reproduces your problem as an issue in telnetlib3 then I'll certainly address it.

But in the case where it is 64 in length, I instead get (of course shortened, but imagine 64 a's) b'echo aaaaaaaa \r\x1b[Auser@...'.

Please note the \r (carriage return) followed by \x1b[A (up) before "user@..." is redrawing the prompt, as I suggested. You have reached the final column and the remote prompt is being redrawn.

A closer look reveals it is not scrolling horizontally, but rather, it has re-written your prompt and continued it to the next line -- it has become a multi-line prompt. That it uses a carriage return is expected, in an interactive terminal, this moves the cursor to the beginning of the line, it does not advance to the next line. It then moves up a row before re-drawing the prompt, this is probably to make enough room at the bottom of the screen to draw 2 lines (and maybe even $PS2 prompt, depending on whether you started a function or quoted character, etc)

The redraw will not happen at multiples of 64, you are now working with a 2-line prompt, the 2nd line probably does not have the PS1 "user@" prompt, the next one will happen at ~77 or so, depending on the continuation character and how much was brought into the next line by word-wrap, etc.

You could patch telnetlib3 directly, to output sent and received characters to a file, or, more easily just use telnetlib3-client CLI directly as an interactive telnet client and record a ttyscript file using https://linux.die.net/man/1/script distributed with tcl-expect package.

Manually perform the same action (or, well, paste a 64-byte character). Make sure to use the same window dimensions, and, you should see the 'effect' that is causing you trouble. Close the connection and "exit" the temporary shell, and you should see the sequences causing you trouble:

 $ script
 Script started, output log file is 'typescript'.
 $ telnetlib3-client example.com
 ...
 hang up/close connection
 $ exit
 Script done.

The final ttypscript file in this working directory can be analyzed with an editor, you should see all of the raw sequences you want to find, or what happens at the 64th character, there

Wrapping a remote prompt is a complex task -- there are many tricks to get it to work reliably that we solved it in the pexpect library, maybe there is code there that can help you https://github.com/pexpect/pexpect/blob/master/pexpect/replwrap.py#L17 -- such as setting a custom PS1 and PS2 prompt to better detect this condition https://github.com/pexpect/pexpect/blob/fc8f062518b40bd0862aae870cdedf5d9c0c7fc3/pexpect/replwrap.py#L122-L124