• web_feed_importer.js erroring out

    From Kurisu@VERT/FINALZON to All on Sun Oct 18 17:09:25 2020
    I just learned of this script thanks to another post here on Dovenet and I went through the process of setting up the modopts feed options and the timed event.

    In attempting to run the event, however, I get error notifications and no joy on data being imported.

    The text in the error log is as follows:

    Sun Oct 18 17:03:15 2020
    evnt FEEDIMP !JavaScript C:\sbbs\exec\web_feed_importer.js line 1: SyntaxError: illegal character

    Sun Oct 18 17:03:15 2020
    evnt FEEDIMP !ERROR 2 (No such file or directory) (WinError 0) in exec.cpp line 668 (sbbs_t::js_execfile) compiling "C:\sbbs\exec\web_feed_importer.js" access=0

    Sun Oct 18 17:03:15 2020
    evnt FEEDIMP Timed event: FEEDIMP returned -1

    I know bugger all of Javascript or much else programming wise, so I can't even begin to debug this myself - my first thoughts lead nowhere so now to ask you fellow hackers for help.

    Thanks.
    _____
    Kurisu Yamato
    www.xadara.com

    ---
    ■ Synchronet ■ Final Zone BBS - finalzone.ddns.net - www.xadara.com
  • From Digital Man@VERT to Kurisu on Sun Oct 18 18:13:40 2020
    Re: web_feed_importer.js erroring out
    By: Kurisu to All on Sun Oct 18 2020 05:09 pm

    I just learned of this script thanks to another post here on Dovenet and I went through the process of setting up the modopts feed options and the timed event.

    In attempting to run the event, however, I get error notifications and no joy on data being imported.

    The text in the error log is as follows:

    Sun Oct 18 17:03:15 2020
    evnt FEEDIMP !JavaScript C:\sbbs\exec\web_feed_importer.js line 1: SyntaxError: illegal character

    What is on line 1 of the file?

    digital man

    Synchronet "Real Fact" #51:
    Rob Swindell was introduced to BBSing in 1982 by his older brother, Dr. Seuss. Norco, CA WX: 77.2°F, 56.0% humidity, 9 mph ENE wind, 0.00 inches rain/24hrs ---
    ■ Synchronet ■ Vertrauen ■ Home of Synchronet ■ [vert/cvs/bbs].synchro.net
  • From echicken@VERT/ECBBS to Kurisu on Sun Oct 18 22:45:33 2020
    Re: web_feed_importer.js erroring out
    By: Kurisu to All on Sun Oct 18 2020 17:09:25

    Sun Oct 18 17:03:15 2020
    evnt FEEDIMP !JavaScript C:\sbbs\exec\web_feed_importer.js line 1: SyntaxError: illegal character

    I know bugger all of Javascript or much else programming wise, so I can't even begin to debug this
    myself - my first thoughts lead nowhere so now to ask you fellow hackers for help.

    Can't really help you without having a look at the feed you're trying to import.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Hatton@VERT/THRCORN to echicken on Mon Oct 19 03:42:07 2020
    Re: web_feed_importer.js erroring out
    By: echicken to Kurisu on Sun Oct 18 2020 10:45 pm

    evnt FEEDIMP !JavaScript C:\sbbs\exec\web_feed_importer.js line 1:
    SyntaxError: illegal character
    Can't really help you without having a look at the feed you're trying to import.

    I'm trying to implement the same thing (curious minds and all) and while he's getting an error I'm not seeing anything happen when I poll https://www.theonion.com/rss

    I can confirm the RSS feed works.

    Hatton

    ... Love conquers all things except poverty and toothache

    ---
    ■ Synchronet ■ Three Corners and Beyond! http://3corners.us
  • From Kurisu@VERT/FINALZON to Digital Man on Sun Oct 18 21:54:32 2020
    Re: web_feed_importer.js erroring out
    By: Digital Man to Kurisu on Sun Oct 18 2020 06:13 pm

    What is on line 1 of the file?

    load("sbbsdefs.js");

    The file is precisely what is in the repository at the moment - I've tried both with and without the commented instructions to identical effect. Even running the script directly via the SBBS control panel results in the same error.
    _____
    Kurisu Yamato
    www.xadara.com

    ---
    ■ Synchronet ■ Final Zone BBS - finalzone.ddns.net - www.xadara.com
  • From Digital Man@VERT to Kurisu on Sun Oct 18 21:47:11 2020
    Re: web_feed_importer.js erroring out
    By: Kurisu to Digital Man on Sun Oct 18 2020 09:54 pm

    Re: web_feed_importer.js erroring out
    By: Digital Man to Kurisu on Sun Oct 18 2020 06:13 pm

    What is on line 1 of the file?

    load("sbbsdefs.js");

    Ah, that line number must be a red herring then.

    digital man

    Rush quote #68:
    He's a radio receiver, tuned to factories and farms .. New World Man
    Norco, CA WX: 67.8°F, 71.0% humidity, 0 mph SE wind, 0.00 inches rain/24hrs
    ---
    ■ Synchronet ■ Vertrauen ■ Home of Synchronet ■ [vert/cvs/bbs].synchro.net
  • From Kurisu@VERT/FINALZON to echicken on Mon Oct 19 02:15:28 2020
    Re: web_feed_importer.js erroring out
    By: echicken to Kurisu on Sun Oct 18 2020 10:45 pm

    Can't really help you without having a look at the feed you're trying to import.

    It appears at least that it actually fails before the import can even happen - even if you clear the [feed] section of modopts it results in the same error. In my rudimentary debugging I considered the feed URL being the issue and tried a few things.

    That being said, the feed I'm looking to import is that of my website, https://www.xadara.com/feed/ Perhaps, of course, a test import or something on a known working system could shed some light...


    _____
    Kurisu Yamato
    www.xadara.com

    ---
    ■ Synchronet ■ Final Zone BBS - finalzone.ddns.net - www.xadara.com
  • From Kurisu@VERT/FINALZON to Digital Man on Mon Oct 19 02:22:29 2020
    Re: web_feed_importer.js erroring out
    By: Digital Man to Kurisu on Sun Oct 18 2020 09:47 pm

    Ah, that line number must be a red herring then.

    Has me for one heck of a loop, honestly.

    Any options to try to run it with a verbose output that may shed light? The original post contains the entire log entry of me running my import command and the error output so naturally we'd need more info.

    Happy to try out whatever options may be available and report back.
    _____
    Kurisu Yamato
    www.xadara.com

    ---
    ■ Synchronet ■ Final Zone BBS - finalzone.ddns.net - www.xadara.com
  • From echicken@VERT/ECBBS to Hatton on Mon Oct 19 10:34:34 2020
    Re: web_feed_importer.js erroring out
    By: Hatton to echicken on Mon Oct 19 2020 03:42:07

    not seeing anything happen when I poll https://www.theonion.com/rss

    When I visit the above in my browser, I'm redirected to 'https://theonion.com/rss'. Remove 'www.' from the URL and try again; maybe it'll work then.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From echicken@VERT/ECBBS to Digital Man on Mon Oct 19 10:51:42 2020
    Re: web_feed_importer.js erroring out
    By: Digital Man to Kurisu on Sun Oct 18 2020 21:47:11

    Ah, that line number must be a red herring then.

    I suspect the error is coming from E4X, so it's 'line 1' of the XML document that's being parsed.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From echicken@VERT/ECBBS to Kurisu on Mon Oct 19 11:01:59 2020
    Re: web_feed_importer.js erroring out
    By: Kurisu to Digital Man on Mon Oct 19 2020 02:22:29

    Ah, that line number must be a red herring then.

    Has me for one heck of a loop, honestly.

    Not sure what to tell you. I have this in modopts.ini:

    [feeds]
    local_rss = https://www.xadara.com/feed/

    web_feed_importer.js imported your feed into my 'local_rss' message base without issue.

    Any options to try to run it with a verbose output that may shed light? The original post contains

    Create a file called rss-test.js in your mods directory, with its contents being:

    load("rss-atom.js");
    new Feed("https://www.xadara.com/feed/");

    Run it via jsexec from a shell or command prompt:

    ../exec/jsexec ./rss-test.js

    (Adjust the above if necessary if you're on Windows.)

    If there's a problem, it should shit out with a verbose error.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From echicken@VERT/ECBBS to Kurisu on Mon Oct 19 11:16:19 2020
    Re: web_feed_importer.js erroring out
    By: Kurisu to echicken on Mon Oct 19 2020 02:15:28

    It appears at least that it actually fails before the import can even happen - even if you clear the
    [feed] section of modopts it results in the same error. In my rudimentary debugging I considered the

    Without a [feeds] section, I would expect web_feed_importer.js to throw 'TypeError: opts is not an object'. So it's not even getting as far as running its main function.

    There's almost certainly a problem with one of the scripts that web_feed_importer is loading, or one of the scripts that one of those scripts is loading, and so forth on down the line.

    I have no such problem on my system, where all of the scripts are up to date as of a few minutes ago.

    Difficult to give specific advice without knowing how you update and manage your system, but you may want to back up your exec/ directory structure and replace it with a fresh copy.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From echicken@VERT/ECBBS to Digital Man on Mon Oct 19 11:16:46 2020
    Re: web_feed_importer.js erroring out
    By: echicken to Digital Man on Mon Oct 19 2020 10:51:42

    I suspect the error is coming from E4X, so it's 'line 1' of the XML document that's being parsed.

    On reflection, probably not. :)

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Kurisu@VERT/FINALZON to echicken on Mon Oct 19 12:57:14 2020
    Re: web_feed_importer.js erroring out
    By: echicken to Kurisu on Mon Oct 19 2020 11:16 am

    Difficult to give specific advice without knowing how you update and manage your system, but you may want to back up your exec/ directory structure and replace it with a fresh copy.

    For full clarity I'm running 3.18b on Windows Server 2008.

    I tried the other posts suggestion of making an rss-test file as instructed and that gave no errors. Given that went okay enough I decided to follow this suggestion replaced all the JS modules in my exec folder with current ones pulled from git and got better results than before - running the timed event no longer results in the previous error, but a new one regarding the server response being "empty."

    I've forgotten exactly what that error said as in the middle of typing this message up to generate that error again I got a DIFFERENT one indicating my message base name was incorrect, which had not shown before. Fixing that typo (I often incorrectly format my message base names when putting them in manually, one day I'll get it right) resulted in the code executing with proper import messages via jsexec.

    I'm not quite sure what I did to get it going. I played around with debugging in jsexec just to see if I could see where the issue crept up (no dice) and following that it just magically worked.

    It certainly seems out of date JS may have been a key factor. In whatever case, it is working now, so, I guess mission complete for the time being? We'll see how it plays out.

    As always I greatly appreciate the help. I wish I didn't have to ask so often but these things elude me at times...

    Wish we could figure out where the actual issue was...
    _____
    Kurisu Yamato
    www.xadara.com

    ---
    ■ Synchronet ■ Final Zone BBS - finalzone.ddns.net - www.xadara.com
  • From echicken@VERT/ECBBS to Kurisu on Mon Oct 19 15:20:42 2020
    Re: web_feed_importer.js erroring out
    By: Kurisu to echicken on Mon Oct 19 2020 12:57:14

    Given that went okay enough I decided to follow this suggestion replaced all the JS modules in my
    exec folder with current ones pulled from git and got better results than before - running the timed

    event no longer results in the previous error, but a new one regarding the server response being
    "empty."

    that error again I got a DIFFERENT one indicating my message base name was incorrect, which had not
    shown before. Fixing that typo (I often incorrectly format my message base names when putting them

    I'm not quite sure what I did to get it going. I played around with debugging in jsexec just to see

    Cleaning up your exec directory likely solved the problem. It's hard to say why that was necessary. A bad update or bad edit in there is the likely culprit.

    If you ever want to modify a script from exec/ or exec/load/, please copy the original script to your mods/ directory and then edit the copy in mods/. This keeps exec/ "clean" and separates your local changes from the stock scripts. More info here:

    http://wiki.synchro.net/dir:mods

    The other errors ("empty response" and the typo) are unrelated to the SyntaxError that you initially
    reported. "Empty response" may be an intermittent problem with the server you're loading the feed from, or it may be an unrelated problem on our end (via http.js).

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Kurisu@VERT/FINALZON to echicken on Mon Oct 19 15:42:29 2020
    Re: web_feed_importer.js erroring out
    By: echicken to Kurisu on Mon Oct 19 2020 03:20 pm

    Cleaning up your exec directory likely solved the problem. It's hard to say why that was necessary. A bad update or bad edit in there is the likely culprit.

    That's what I'm suspecting. I had to kind of bastardize 3.18b onto my setup... ah, the learning process.

    If you ever want to modify a script from exec/ or exec/load/, please copy the original script to your mods/ directory and then edit the copy in mods/. This keeps exec/ "clean" and separates your local changes from the stock scripts. More info here:

    http://wiki.synchro.net/dir:mods

    I really must take advantage of the mods directory. Never had the best workflow with these kinds of things. Could well be what caused the original improper files to be lurking around, as I may have avoided updating something I should have, like a fool.

    Now if only I could figure out enough syntaxt to make my feeds from my blog not be so ugly... but that's it's own thing. The function is working fine for me now, it would seem, so I'll get to implimenting the random ideas I had for it.

    Thanks again.
    _____
    Kurisu Yamato
    www.xadara.com

    ---
    ■ Synchronet ■ Final Zone BBS - finalzone.ddns.net - www.xadara.com
  • From Bob Roberts@VERT/HOVAL to echicken on Mon Oct 19 14:28:11 2020
    Re: web_feed_importer.js erroring out
    By: echicken to Kurisu on Mon Oct 19 2020 11:16 am

    There's almost certainly a problem with one of the scripts that web_feed_importer is loading, or one of the scripts that one
    of those scripts is loading, and so forth on down the line.

    I have no such problem on my system, where all of the scripts are up to date as of a few minutes ago.

    If it makes anyone feel any better, I'm having no problems pulling in RSS headlines from CNN on my BBS. However the feeds from Politico result in an error. So if one feed doesn't work... maybe another will.

    |08~|05B|03ob|08:|06R|03ob|08~
    |07



    ... No good deed goes unpunished.

    ---
    ■ Synchronet ■ Halls of Valhalla <> San Francisco <> hovalbbs.com
  • From Hatton@VERT/THRCORN to echicken on Mon Oct 19 19:33:00 2020
    not seeing anything happen when I poll https://www.theonion.com/rss

    When I visit the above in my browser, I'm redirected to 'https://theonion.com/rss'. Remove 'www.' from the URL and try again; maybe it'll work then.

    When I do that I get a log entry "Empty Response from Server"

    Could the SSL address be throwing things off?

    Hatton


    ... Support bacteria! It's the only culture some people have.
    --- MultiMail/Win v0.52
    ■ Synchronet ■ Three Corners and Beyond! http://3corners.us
  • From Tracker1@VERT/TRN to Digital Man on Mon Oct 19 20:20:56 2020
    On 10/18/2020 6:13 PM, Digital Man wrote:
    evnt FEEDIMP !JavaScript C:\sbbs\exec\web_feed_importer.js line 1:
    SyntaxError: illegal character

    What is on line 1 of the file?

    Guessing it's a BOM character sequence if loaded in a windows text
    editor and re-saved.

    --
    Michael J. Ryan
    tracker1 +o Roughneck BBS

    ---
    ­ Synchronet ­ Roughneck BBS - coming back 2/2/20
  • From echicken@VERT/ECBBS to Bob Roberts on Tue Oct 20 00:09:07 2020
    Re: web_feed_importer.js erroring out
    By: Bob Roberts to echicken on Mon Oct 19 2020 14:28:11

    BBS. However the feeds from Politico result in an error. So if one feed doesn't work... maybe

    If you want to share the URL to a particular Politico feed that isn't working, I can take a look.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From echicken@VERT/ECBBS to Hatton on Tue Oct 20 00:11:09 2020
    Re: web_feed_importer.js erro
    By: Hatton to echicken on Mon Oct 19 2020 19:33:00

    'https://theonion.com/rss'. Remove 'www.' from the URL and try again;

    When I do that I get a log entry "Empty Response from Server"

    Could the SSL address be throwing things off?

    Not sure; same thing is happening to me here. HTTPS shouldn't be a problem, but I'll have to take a closer look and see what's up.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From echicken@VERT/ECBBS to Hatton on Tue Oct 20 00:15:34 2020
    Re: web_feed_importer.js erro
    By: echicken to Hatton on Tue Oct 20 2020 00:11:09

    'https://theonion.com/rss'. Remove 'www.' from the URL and try again;

    When I do that I get a log entry "Empty Response from Server"

    Could the SSL address be throwing things off?

    Not sure; same thing is happening to me here. HTTPS shouldn't be a problem, but I'll have to take a
    closer look and see what's up.

    Okay, never mind - my browser was misleading me, and the 'www' is required after all.

    This little test script, run via jsexec, is able to load the feed and print some articles to my terminal. Please give it a try and let me know what you see:

    load("rss-atom.js");
    var f = new Feed("https://www.theonion.com/rss");
    for(var c = 0; c < f.channels.length; c++) {
    writeln(f.channels[c].title);
    writeln(f.channels[c].updated);
    for(var i = 0; i < f.channels[c].items.length; i++) {
    writeln(f.channels[c].items[i].title);
    writeln(f.channels[c].items[i].author);
    writeln(f.channels[c].items[i].date);
    writeln(f.channels[c].items[i].body);
    writeln("---");
    }
    writeln("---");
    }

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Hatton@VERT/THRCORN to echicken on Tue Oct 20 11:25:14 2020
    This little test script, run via jsexec, is able to load the feed and
    print
    some articles to my terminal. Please give it a try and let me know what
    you
    see:

    Looking at the output below and comparing it to the actual XML output, I
    think the culprit here lies in elements that are expressed with "CDATA".

    Also the code references the "body" element and that does not exist in this feed, they use "description" instead.

    Reading script from /sbbs/exec/rsstest.js
    /sbbs/exec/rsstest.js compiled in 0.00 seconds
    The Onion





    ---




    ---
    <snipped for ease of reading>
    /sbbs/exec/rsstest.js executed in 0.18 seconds

    JavaScript: Destroying context
    JavaScript: Destroying runtime
    ---
    ■ Synchronet ■ Three Corners and Beyond! http://3corners.us
  • From echicken@VERT/ECBBS to Hatton on Tue Oct 20 12:02:17 2020
    Re: Re: web_feed_importer.js erro
    By: Hatton to echicken on Tue Oct 20 2020 11:25:14

    Also the code references the "body" element and that does not exist in this feed, they use "description" instead.

    That's fine - "body" is a property of the Item object, which is produced when parsing a feed. It's derived from an item's "description" (RSS) or "summary" (Atom) element. It's not a direct reference to an XML element.

    Looking at the output below and comparing it to the actual XML output, I think the culprit here lies in elements that are expressed with "CDATA".

    When I run that script on my system, all of the CDATA-enclosed content appears to be parsed out properly. I don't have blank lines in my output (except for 'author', which appears to be absent from their items).

    We're seeing different results, so there's some difference in the code we're running on our systems. How up to date is your BBS, including the scripts in exec/ and exec/load/?

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Hatton@VERT/THRCORN to echicken on Tue Oct 20 17:20:43 2020
    Re: Re: web_feed_importer.js erro
    By: echicken to Hatton on Tue Oct 20 2020 12:02 pm

    We're seeing different results, so there's some difference in the code we're running on our systems. How up to date is your BBS, including the scripts in exec/ and exec/load/?

    I thought I was running latest version, I'll do a git pull and rebuild (with symlinks) and see what happens.

    Hatton

    ... File not found, I'll load something *I* think is interesting.

    ---
    ■ Synchronet ■ Three Corners and Beyond! http://3corners.us
  • From Hatton@VERT/THRCORN to echicken on Tue Oct 20 17:45:40 2020
    Re: Re: web_feed_importer.js erro
    By: Hatton to echicken on Tue Oct 20 2020 05:20 pm

    We're seeing different results, so there's some difference in the
    code we're running on our systems. How up to date is your BBS,
    including the scripts in exec/ and exec/load/?

    I thought I was running latest version, I'll do a git pull and rebuild (with symlinks) and see what happens.

    That was the issue - I've been doing GIT pulls and rebuilds but have not been copying over the updated JS files from exec and exec/load.

    Thanks!
    Hatton

    ... Men will always be men -- no matter where they are.

    ---
    ■ Synchronet ■ Three Corners and Beyond! http://3corners.us
  • From Bob Roberts@VERT/HOVAL to echicken on Tue Oct 20 12:29:52 2020
    Re: web_feed_importer.js erroring out
    By: echicken to Bob Roberts on Tue Oct 20 2020 12:09 am

    BBS. However the feeds from Politico result in an error. So if one
    feed doesn't work... maybe

    If you want to share the URL to a particular Politico feed that isn't working, I can take a look.

    Well, I tried it again and it's working now. So perhaps a temporary error. I'm now importing 6-7 different feeds into 3 different local message bases, allowing me to use SBBS as a bit of an RSS reader, which is fun.

    It would be fun if there was a modopts.ini option that would allow you to specify the "from" name for each feed. The autogenerated names get kinda ugly.

    |08~|05B|03ob|08:|06R|03ob|08~
    |07



    ... I want to write a mystery novel...or do I?

    ---
    ■ Synchronet ■ Halls of Valhalla <> San Francisco <> hovalbbs.com
  • From echicken@VERT/ECBBS to Bob Roberts on Tue Oct 20 16:09:23 2020
    Re: web_feed_importer.js erroring out
    By: Bob Roberts to echicken on Tue Oct 20 2020 12:29:52

    It would be fun if there was a modopts.ini option that would allow you to specify the "from" name for each feed. The
    autogenerated names get kinda ugly.

    Unfortunately the current configuration scheme doesn't allow for that sort of thing. The script either uses the contents of the 'author' field for each item as the 'from' address, or if that's empty/absent, uses 'Web Feed Importer'.

    I'd really like to rewrite this script since there are a lot of things I'd do better/differently now - but I made some poor choices when I wrote it, so I'll need to be careful to avoid it creating a bunch of dupe messages.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Bob Roberts@VERT/HOVAL to echicken on Tue Oct 20 12:51:12 2020
    Re: web_feed_importer.js erro
    By: echicken to Hatton on Tue Oct 20 2020 12:15 am

    This little test script, run via jsexec, is able to load the feed and print some articles to my terminal. Please give it a try and let me know what you see:

    I'm having some problems with https://www.theverge.com/rss/full.xml

    So I'm using your test script, and nothing is coming out. But when I load it in Firefox the XML displays OK.

    Loading configuration files from /sbbs/ctrl
    JavaScript-C 1.8.5 2011-03-31
    JavaScript: Creating runtime: 8388608 bytes
    JavaScript: Initializing context (stack: 16384 bytes)

    Reading script from /home/sbbs/sbbs/mods/rss-test.js /home/sbbs/sbbs/mods/rss-test.js compiled in 0.00 seconds


    ---
    /home/sbbs/sbbs/mods/rss-test.js executed in 0.15 seconds

    |08~|05B|03ob|08:|06R|03ob|08~
    |07



    ... Hello, he lied.
    ---
    ■ Synchronet ■ Halls of Valhalla <> San Francisco <> hovalbbs.com
  • From Kurisu@VERT/FINALZON to echicken on Tue Oct 20 17:00:22 2020
    Re: web_feed_importer.js erroring out
    By: echicken to Bob Roberts on Tue Oct 20 2020 04:09 pm

    It would be fun if there was a modopts.ini option that would allow you to specify the "from" name for each feed. The
    autogenerated names get kinda ugly.

    Unfortunately the current configuration scheme doesn't allow for that sort of thing. The script either uses the contents of the 'author' field for each item as the 'from' address, or if that's empty/absent, uses 'Web Feed Importer'.

    I'd really like to rewrite this script since there are a lot of things I'd do better/differently now - but I made some poor choices when I wrote it, so I'll need to be careful to avoid it creating a bunch of dupe messages.

    To circumvent some of the nasty I spent much of last night making a custom RSS feed for my website to try to pull things a LITTLE better. It's one hell of a kludge and requires a very particular combination of Wordpress hack + constant polling to really make it "sane" in the script, but it at least allowed for things like article names (rather than just "Xadara" for everything) but again, you have to really force it to pull that info and do posts one-at-a-time... so it wound up being something I wasn't happy with.

    Kind of makes it annoying to use on my site for now, but hey, it works great for a weather feed! Now if only I could find one that didn't update every 10 minutes, lol...

    If it ever gets to being rewritten I'd LOVE to put it to its paces for importing content from a Wordpress site and formatting them reasonably well, since of course any updated script would wind up being more intricate, as the above request from Bob hints at.

    Hell, if I could code worth a damn I'd try to improve it, but all the logic in that (obviously) eludes me when it comes to pulling X value to Y area.

    What I'm getting at is for multi-faceted BBS sysops, like me, such is already a useful enough idea to start with and could mature quite a bit... but, priorities. I could always get off my ass and figure it out myself, right?

    Bah, I'm just thinking out loud. Consider this a supportive post of "if you ever want to try to write a new version of it, I'd be willing to support as best I can with testing in my particular use cases which seem unique enough here."
    _____
    Kurisu Yamato
    www.xadara.com

    ---
    ■ Synchronet ■ Final Zone BBS - finalzone.ddns.net - www.xadara.com
  • From echicken@VERT/ECBBS to Bob Roberts on Tue Oct 20 22:29:03 2020
    Re: web_feed_importer.js erro
    By: Bob Roberts to echicken on Tue Oct 20 2020 12:51:12

    I'm having some problems with https://www.theverge.com/rss/full.xml

    So I'm using your test script, and nothing is coming out. But when I load it in Firefox the XML displays OK.

    Same result here. It's an Atom feed rather than RSS - which should work, but I haven't tested against one in a long time. Will see if I can figure it out.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From echicken@VERT/ECBBS to Bob Roberts on Tue Oct 20 23:28:49 2020
    Re: web_feed_importer.js erro
    By: echicken to Bob Roberts on Tue Oct 20 2020 22:29:03

    I'm having some problems with https://www.theverge.com/rss/full.xml

    I've just pushed a couple of changes to exec/load/rss-atom.js which should help. A little more work may be needed; it's possible that not all of the fields (title, author, date, etc.) are being handled correctly for this type of feed. Too tired to dig deeper right now; please let me know if you see anything weird.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Bob Roberts@VERT/HOVAL to echicken on Wed Oct 21 13:19:10 2020
    Re: web_feed_importer.js erro
    By: echicken to Bob Roberts on Tue Oct 20 2020 11:28 pm

    I've just pushed a couple of changes to exec/load/rss-atom.js which should help. A little more work may be needed; it's possible that not all of the fields (title, author, date, etc.) are being handled correctly for this type of feed. Too tired to dig deeper right now; please let me know if you see anything weird.

    Yup. After pulling the new rss-atom.js down the feed from theverge works pretty good. Thanks!


    |08~|05B|03ob|08:|06R|03ob|08~
    |07



    ... If you can't learn to do it well, learn to enjoy doing it badly.

    ---
    ■ Synchronet ■ Halls of Valhalla <> San Francisco <> hovalbbs.com
  • From Bob Roberts@VERT/HOVAL to echicken on Tue Oct 27 22:59:04 2020
    Re: web_feed_importer.js erro
    By: echicken to Bob Roberts on Tue Oct 20 2020 11:28 pm

    I've just pushed a couple of changes to exec/load/rss-atom.js which should help. A little more work may be needed; it's possible that not all of the fields (title, author, date, etc.) are being handled correctly for this type of feed. Too tired to dig deeper right now; please let me know if you see anything weird.

    Hi there,

    I'm getting periodic crashes with this updated script. It happens occasionally perhaps due to the content of one of the RSS feeds. When it happens SBBS dies completely, then restarts itself. There isn't an error in the System Journal other then lack of exit from RSSIN, then I see all the reload messages.

    Is there something I can do that might provide more info on the crash?

    |08~|05B|03ob|08:|06R|03ob|08~
    |07



    ... This tagline's just for you.

    ---
    ■ Synchronet ■ Halls of Valhalla <> San Francisco <> hovalbbs.com
  • From echicken@VERT/ECBBS to Bob Roberts on Wed Oct 28 10:43:36 2020
    Re: web_feed_importer.js erro
    By: Bob Roberts to echicken on Tue Oct 27 2020 22:59:04

    I'm getting periodic crashes with this updated script. It happens occasionally perhaps due to the content of one of the RSS feeds. When it happens SBBS dies completely, then restarts itself. There isn't an error in the System Journal other then lack of exit from RSSIN, then I see all the reload messages.

    Is there something I can do that might provide more info on the crash?

    Probably not. I'd have to add a bunch of debug logging to the script so that we might figure out exactly when/why this is happening.

    If you noticed that this started after you added a particular feed, you could disable that feed for the time being.

    I'm thinking about creating a new project to handle viewing / importing feeds, but I probably won't get to that for a few weeks.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Digital Man@VERT to Bob Roberts on Wed Oct 28 09:59:16 2020
    Re: web_feed_importer.js erro
    By: Bob Roberts to echicken on Tue Oct 27 2020 10:59 pm

    Re: web_feed_importer.js erro
    By: echicken to Bob Roberts on Tue Oct 20 2020 11:28 pm

    I've just pushed a couple of changes to exec/load/rss-atom.js which should help. A little more work may be needed; it's possible that not all of the fields (title, author, date, etc.) are being handled correctly for this type of feed. Too tired to dig deeper right now; please let me know if you see anything weird.

    Hi there,

    I'm getting periodic crashes with this updated script. It happens occasionally perhaps due to the content of one of the RSS feeds. When it happens SBBS dies completely, then restarts itself. There isn't an error in the System Journal other then lack of exit from RSSIN, then I see all the reload messages.

    Is there something I can do that might provide more info on the crash?

    This would be helpful:
    http://wiki.synchro.net/howto:gdb
    --
    digital man

    Sling Blade quote #15:
    Doyle Hargraves: What'cha doin' with that lawn mower blade Karl?
    Norco, CA WX: 65.1°F, 33.0% humidity, 0 mph SSW wind, 0.00 inches rain/24hrs ---
    ■ Synchronet ■ Vertrauen ■ Home of Synchronet ■ [vert/cvs/bbs].synchro.net
  • From Bob Roberts@VERT/HOVAL to echicken on Wed Oct 28 08:32:21 2020
    Re: web_feed_importer.js erro
    By: echicken to Bob Roberts on Wed Oct 28 2020 10:43 am

    Probably not. I'd have to add a bunch of debug logging to the script so that we might figure out exactly when/why this is happening.

    I'll try to figure out what feed it is, and let you know. My suspicion it's a missing xml tag or something that periodically happens based on whatever content they are running at the time.

    I'm thinking about creating a new project to handle viewing / importing feeds, but I probably won't get to that for a few weeks.

    That would be cool. Not sure if you're thinking of creating an external RSS "door", but I really like having them imported into the message bases, as it creates a nice integrated feel and gives at least the illusion of activity. Most RSS feeds integrate pretty cleanly, some have extra spaces or funny characters, but overall it works well.

    |08~|05B|03ob|08:|06R|03ob|08~
    |07



    ... The average raindrop falls at 7 miles per hour.

    ---
    ■ Synchronet ■ Halls of Valhalla <> San Francisco <> hovalbbs.com
  • From echicken@VERT/ECBBS to Bob Roberts on Wed Oct 28 14:49:46 2020
    Re: web_feed_importer.js erro
    By: Bob Roberts to echicken on Wed Oct 28 2020 08:32:21

    I'll try to figure out what feed it is, and let you know. My suspicion it's a missing xml tag or something that periodically happens based on whatever content they are running at the time.

    I think that's unlikely - you'd probably just see an error message which might hint at a failed attempt at XML parsing, but the BBS is unlikely to crash. But maybe.

    I lean toward there being some unexpected characters in an item title or something like that, causing a problem when an attempt is made to save the message. Or some massive amount of memory being used, something like that.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From echicken@VERT/ECBBS to Bob Roberts on Wed Oct 28 14:55:26 2020
    Re: web_feed_importer.js erro
    By: Bob Roberts to echicken on Wed Oct 28 2020 08:32:21

    That would be cool. Not sure if you're thinking of creating an external RSS "door", but I really like having them imported into the message bases,

    I am thinking of that, but it would still be backed by a message base. There would be an RSS reader module that you could browse feeds from, but ultimately it's pulling data out of message bases. You could hide those bases from your users or not, or use the reader module or not. Could be a news ticker or scrolling newsreader, that sort of thing.

    In reality it's just a chance for me to sidestep some problems with web_feed_importer by offering an alternative. There are certain things that make revising or overhauling the existing script a bit problematic.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Bob Roberts@VERT/HOVAL to Digital Man on Wed Oct 28 13:45:33 2020
    Re: web_feed_importer.js erro
    By: Digital Man to Bob Roberts on Wed Oct 28 2020 09:59 am

    This would be helpful:
    http://wiki.synchro.net/howto:gdb

    Okay, I think I've got the configuration done for core dumps, so if it happens again I'll try the gdb steps.

    |08~|05B|03ob|08:|06R|03ob|08~
    |07

    ---
    ■ Synchronet ■ Halls of Valhalla <> San Francisco <> hovalbbs.com
  • From Digital Man@VERT to Bob Roberts on Wed Oct 28 16:28:36 2020
    Re: web_feed_importer.js erro
    By: Bob Roberts to Digital Man on Wed Oct 28 2020 01:45 pm

    Re: web_feed_importer.js erro
    By: Digital Man to Bob Roberts on Wed Oct 28 2020 09:59 am

    This would be helpful:
    http://wiki.synchro.net/howto:gdb

    Okay, I think I've got the configuration done for core dumps, so if it happens again I'll try the gdb steps.

    Appreciate it,
    --
    digital man

    Rush quote #50:
    I've always been a big sponge. - Neil Peart
    Norco, CA WX: 74.6°F, 32.0% humidity, 8 mph NE wind, 0.00 inches rain/24hrs
    ---
    ■ Synchronet ■ Vertrauen ■ Home of Synchronet ■ [vert/cvs/bbs].synchro.net
  • From Gamgee@VERT/PALANT to echicken on Wed Oct 28 20:48:00 2020
    echicken wrote to Bob Roberts <=-

    Re: web_feed_importer.js erro
    By: Bob Roberts to echicken on Wed Oct 28 2020 08:32:21

    That would be cool. Not sure if you're thinking of creating an external RSS "door", but I really like having them imported into the message bases,

    I am thinking of that, but it would still be backed by a message
    base. There would be an RSS reader module that you could browse
    feeds from, but ultimately it's pulling data out of message
    bases. You could hide those bases from your users or not, or use
    the reader module or not. Could be a news ticker or scrolling
    newsreader, that sort of thing.

    I'd be quite interested in this also, and I'll thank you in
    advance for considering it.



    ... A day without sunshine is like night.
    --- MultiMail/Linux v0.52
    ■ Synchronet ■ Palantir BBS * palantirbbs.ddns.net * Pensacola, FL
  • From Sneaky@VERT/CHOICE to All on Tue Nov 3 18:22:20 2020
    Re: web_feed_importer.js erroring out
    By: Kurisu to Digital Man on Mon Oct 19 2020 02:22 am

    Hi all

    I have setup web_feed_importer.js and the one that was there when I setup
    works fine + another one I added to the list.

    I have been searching for rss feed on the internet and I like
    to ask if you could list a few rss feed that I could use on my bbs
    just all topic and bbs stuff if any

    Thank you

    Ian S 1st Choice Core Nz

    ---
    ■ Synchronet ■ 1st Choice Core - 1stchoicecore.co.nz
  • From Bob Roberts@VERT/HOVAL to Sneaky on Tue Nov 3 11:20:35 2020
    Re: web_feed_importer.js
    By: Sneaky to All on Tue Nov 03 2020 06:22 pm

    I have been searching for rss feed on the internet and I like
    to ask if you could list a few rss feed that I could use on my bbs
    just all topic and bbs stuff if any

    Here's what I'm using. Still playing with it. I try to find RSS feeds that have full article text, but they are harder and harder to find, especially with the news sites.

    LOCAL-HEADLINES = http://rss.cnn.com/rss/edition.rss,http://feeds.bbci.co.uk/news/rss.xml
    LOCAL-POLITICS = https://rss.politico.com/politicopulse.xml
    LOCAL-TECHNEWS = https://www.techmeme.com/feed.xml?x=1,https://TechCrunch.com/feed/,http://feeds .arstechnica.com/arstechnica/index,https://www.theverge.com/rss/full.xml

    |01bobbobbobbob|09bob|03bob|11bob|03bob|09bob|01bobbobbob |01robrobrobrob|09rob|03rob|11rob|03rob|09rob|01robrobrob

    ---
    ■ Synchronet ■ Halls of Valhalla <> San Francisco <> hovalbbs.com
  • From echicken@VERT/ECBBS to Bob Roberts on Wed Nov 4 00:30:01 2020
    Re: web_feed_importer.js
    By: Bob Roberts to Sneaky on Tue Nov 03 2020 11:20:35

    Here's what I'm using. Still playing with it. I try to find RSS feeds that have full article text, but they are harder and harder to find, especially with the news sites.

    They generally want you to click through for the full story, so they can serve more ads and otherwise monetize your reading. RSS is still in use, but just isn't as good as it was 10+ years ago.

    ---
    echicken
    electronic chicken bbs - bbs.electronicchicken.com
    ■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com
  • From Sneaky@VERT/CHOICE3 to Bob Roberts on Thu Nov 5 08:35:00 2020
    Hi Bob Roberts

    to ask if you could list a few rss feed that I could use on my bbs
    just all topic and bbs stuff if any

    Here's what I'm using. Still playing with it. I try to find RSS feeds that have full article text, but they are harder and harder to find, especially with the news sites.

    yes they are hard to find, but well keep searching.

    LOCAL-HEADLINES = http://rss.cnn.com/rss/edition.rss,http://feeds.bbci.co.uk/news/rss.xml LOCAL-POLITICS = https://rss.politico.com/politicopulse.xml
    LOCAL-TECHNEWS = https://www.techmeme.com/feed.xml?x=1,https://TechCrunch.com/feed/,http: //feeds .arstechnica.com/arstechnica/index,https://www.theverge.com/rss/full.xml


    Thank you for there ones, I have a http://rss.cnn.com now and probably add more rss feed
    from there, with the one's above.

    Thank you for your reply

    Ian S 1st Choice Core Nz

    --- MultiMail/Win v0.52
    ■ Synchronet ■ 4th Choice Core/New Zealand