Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error with saspy.read_csv() when passing an URL for a filename #634

Open
mtalanay opened this issue Feb 22, 2025 · 7 comments
Open

Error with saspy.read_csv() when passing an URL for a filename #634

mtalanay opened this issue Feb 22, 2025 · 7 comments

Comments

@mtalanay
Copy link

Describe the bug
I am trying to import a CSV file which is an URL link using read_csv() to create a SAS dataset. There are few errors reported in the log and may be primary one is from the filename statement which has the URL enclosed in double quotes causing "&" to be resolved.

Access Method         = STDIO
SAS Config name       = default
SAS Config file       = /usr/local/lib/python3.11/site-packages/saspy/sascfg.py
WORK Path             = /tmp/SAS_workD6AC00005477_run-67b9d589ef61b605d4f00193-q9llf/
SAS Version           = V.04.00M0P09162024
SASPy Version         = 5.101.0
Teach me SAS          = False
Batch                 = False
Results               = Pandas
SAS Session Encoding  = utf-8
Python Encoding value = utf_8
SAS process Pid value = 21623
SASsession started    = Sat Feb 22 19:10:44 2025

To Reproduce
Details below:

filename _x url 22 ! "https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

The URL string has few % and & - here is one piece of URL "GKl9eqTlImvzYS66LqIewbx9wmm6qnkigD1BQP7YkpescyTnLjd0%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20250222T200156Z&X-Amz-Sign"

NOTE: The quoted string currently being processed has become more than 262 bytes long. You might have unbalanced quotation marks. WARNING: Apparent symbolic reference X not resolved. WARNING: Apparent symbolic reference X not resolved. WARNING: Apparent symbolic reference X not resolved. WARNING: Apparent symbolic reference X not resolved. WARNING: Apparent symbolic reference X not resolved. WARNING: Apparent symbolic reference X not resolved. WARNING: Apparent symbolic reference X not resolved. 23 proc import datafile=_x out='ae_aim'n dbms=csv replace; getnames=YES; run;filename _x clear;

Appreciate any help!
@tomweber-sas
Copy link
Contributor

Hey, that looks like a parsing issue on the SAS side. It seems to be thinking there are macros and such embedded in there or something. I'll have to see what you are actually submitting and the log with all the output so I can see what's really going on. Can you submit the same code as SASPy is generating in a stand alone session in a way that works with no errors from SAS?
If you don't want to post the real code/output here (confidential or anything), you can just email it to me instead.

@tomweber-sas
Copy link
Contributor

Well, you mentioned double quotes in your comment, so I just tried a simple case with single vs double quotes. I see with single it doesn't complain and w/ double it tries to resolve embedded macros. We are so inconsistent sometimes w/ parsing and quoting an macro language; drives me nuts.
Does your case work if it's just single quotes around your url? I can easily change double quotes to single for the code gen in the method. Then that begs the question of if anyone is counting on this to use a macro in the string and want's it to get resolved.
If single quotes on the filename works for your case, I'll figure out how to resolve this. If that's not all there is to it, I'll need more info like I was asking for, to be sure I understand the problem.
Tom

  6? filename x url "xxx&booxxx%32xxxx";

WARNING: Apparent symbolic reference BOOXXX not resolved.

  7? filename x url 'xxx&booxxx%32xxxx';
  8? filename x list;

NOTE: Fileref= X
      Physical Name= xxx&booxxx%32xxxx

  9?

@tomweber-sas
Copy link
Contributor

As this does seem to be an issue w/ double vs. sincgle quotes, I've pushed the change to generate the path w/ single quotes and see it behave better. Can you pull from main and test it out with your scenario? It seems to resolve the issue from what I can tell with my test cases. Needs to work for you too though. You can pull from main via
pip install git+https://[email protected]/sassoftware/saspy.git@branchname
or since main is the default branch, you don't need @main:
pip install git+https://[email protected]/sassoftware/saspy.git

>>> sd = sas.read_csv("http://xxx&booxxx%32xxxx")
/opt/tom/github/saspy/saspy/sasiostdio.py:1118: UserWarning: Noticed 'ERROR:' in LOG, you ought to take a look and see if there was a problem
  warnings.warn("Noticed 'ERROR:' in LOG, you ought to take a look and see if there was a problem")
>>> print(sas.lastlog())

21   ;*';*";*/;
22   filename _x url 'http://xxx&booxxx%32xxxx';
23   proc import datafile=_x out='_csv'n dbms=csv replace;  run;filename _x clear;
NOTE: Unable to connect to host xxx&booxxx%32xxxx. Check validity of host name.
ERROR: Physical file does not exist, http://xxx&booxxx%32xxxx.
ERROR: Import unsuccessful.  See SAS Log for details.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE IMPORT used (Total process time):
      real time           2.82 seconds
      cpu time            0.36 seconds

NOTE: Fileref _X has been deassigned.
24
25
26   ;*';*";*/;%put %upcase(e3969440a681a2408885998500000004);
E3969440A681A2408885998500000004
27   ;*';*";*/;
28   data _null_; e = exist("'_csv'n");
29   v = exist("'_csv'n", 'VIEW');
30    if e or v then e = 1;
31   put 'TABLE_EXISTS=' e 'TAB_EXTEND=';run;
TABLE_EXISTS=0 TAB_EXTEND=
NOTE: DATA statement used (Total process time):
      real time           0.06 seconds
      cpu time            0.02 seconds

32
33   ;*';*";*/;%put %upcase(e3969440a681a2408885998500000005);
E3969440A681A2408885998500000005
>>> sas.submitLOG('''filename _x url 'http://xxx&booxxx%32xxxx';''')

34   ods listing close;ods html5 (id=saspy_internal) file=stdout options(bitmap_mode='inline') device=svg style=HTMLBlue; ods
34 ! graphics on / outputfmt=png;
NOTE: Writing HTML5(SASPY_INTERNAL) Body file: STDOUT
35
36   filename _x url 'http://xxx&booxxx%32xxxx';
37
38   ods html5 (id=saspy_internal) close;ods listing;

>>> sas.submitLOG('''filename _x url "http://xxx&booxxx%32xxxx";''')

40   ods listing close;ods html5 (id=saspy_internal) file=stdout options(bitmap_mode='inline') device=svg style=HTMLBlue; ods
40 ! graphics on / outputfmt=png;
NOTE: Writing HTML5(SASPY_INTERNAL) Body file: STDOUT
41
42   filename _x url "http://xxx&booxxx%32xxxx";
WARNING: Apparent symbolic reference BOOXXX not resolved.
43
44   ods html5 (id=saspy_internal) close;ods listing;

>>>

@mtalanay
Copy link
Author

mtalanay commented Feb 24, 2025 via email

@tomweber-sas
Copy link
Contributor

That great! Thanks for trying it out so quickly. I see I have the same issue in write_csv, but everywhere else in saspy that I generate the filename statement I use single quotes. So, I fixed it for write_csv also. I have one other change at main for an internal use that has a strange issue. Once I decide if I want that change in or not, I'll build a new release with this for for you. I'll keep this open till I do that, then I expect it can be closed once that done?
Thanks,
Tom

@mtalanay
Copy link
Author

Sure Tom - that should be fine. for now we are proceeding with the submit option. thanks.

sas.symput('csvlink', link)

sas.submitLOG(
"""
options noquotelenmax;
filename _x url "&csvlink";
proc import datafile=_x out=work.csvdata replace dbms=csv; getnames=yes; run;
"""
)

@tomweber-sas
Copy link
Contributor

Yep, that's the other workaround where you take advantage of it parsing the macro from the string!
In fact you can use the read_csv() method after you symput the macro (with the code that's using the double quotes - w/out the fix):

sas.symput('csvlink', link)
sd = sas.read_csv("&csvlink")

I'll try to get this fix for you into a new release this week so you don't have to do either of these :)
Tom

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tomweber-sas @mtalanay