• Newslink.js filtering

    From plt@VERT/SBBS to All on Tue Nov 10 15:41:45 2020
    I know there has been some debate about my request to adding a feature to be able to filter within the body of the newsgroups. Its a option, if you do not like then do not use it. Filtering out the good and bad messages using only the subject line does not always work. Some words being filter within the body of the email for example porn, sex or horney and this type of material should be on your bbs system in the first place. Filtering by the subject should remain as a option, but adding a second filtering option is better. It givves the tools the sysop need to do their job and takes very small effort to do it.

    ---
    ■ Synchronet ■ sbbs.dynu.net 2025
  • From Tracker1@VERT/HUB to plt on Tue Nov 10 13:49:02 2020
    On 11/10/2020 8:41 AM, plt wrote:
    I know there has been some debate about my request to adding a feature to be able to filter within the body of the newsgroups. Its a option, if you do not like then do not use it. Filtering out the good and bad messages using only the subject line does not always work. Some words being filter within the body of the email for example porn, sex or horney and this type of material should be on your bbs system in the first place. Filtering by the subject should remain as a option, but adding a second filtering option is better. It givves the tools the sysop need to do their job and takes very small effort to do it.

    I was thinking it might be worthwhile to have a generic option to run an application or script against two files (headers and body) for messages
    being imported via qwk, ftn or newslink as well as smtp.

    Not sure if there's already something built in for this... and if there
    is, apologies. Just thinking something pretty generic could be used for
    this and beneficial. With the most generic action, of if either file is deleted, then it doesn't post, and puts a message out. This could be
    used to support virus scans, etc as well.

    --
    Michael J. Ryan
    tracker1 +o Roughneck BBS
    ---
    ­ Synchronet ­ The Hub - http://hub.bbs.io/
  • From Mortifis@VERT/ALLEYCAT to Tracker1 on Tue Nov 10 17:38:30 2020
    On 11/10/2020 8:41 AM, plt wrote:

    I was thinking it might be worthwhile to have a generic option to run an application or script against two files (headers and body) for messages being imported via qwk, ftn or newslink as well as smtp.


    The only real issue I see with filtering the message body for key words is that 'innocent' messages (conversations) could get filtered out, thereby failing to deliver someone's reply ... so, even if this feature were implemented (which would have to be strict in its rules) I would not use it ... just sayin

    ~Mortifis

    ---
    ­ Synchronet ­ AlleyCat! BBS Lake Echo, NS Canada
  • From MRO@VERT/BBSESINF to Tracker1 on Tue Nov 10 16:12:10 2020
    Re: Re: Newslink.js filtering
    By: Tracker1 to plt on Tue Nov 10 2020 01:49 pm

    I was thinking it might be worthwhile to have a generic option to run an application or script against two files (headers and body) for messages being imported via qwk, ftn or newslink as well as smtp.

    Not sure if there's already something built in for this... and if there is, apologies. Just thinking something pretty generic could be used for this and beneficial. With the most generic action, of if either file is deleted, then it doesn't post, and puts a message out. This could be
    used to support virus scans, etc as well.

    i think years ago i asked rob how to filter newsgroup msgs for certain spam by editing the .js file. i forgot how to do it and dont have the old .js file.

    anyways, if you did the 2 different files and ran scripts on them it would really slow down msg import. depends on if you care about that.
    ---
    ■ Synchronet ■ ::: BBSES.info - free BBS services :::
  • From plt@VERT/SBBS to Tracker1 on Tue Nov 10 19:18:08 2020
    Re: Re: Newslink.js filtering
    By: Tracker1 to plt on Tue Nov 10 2020 13:49:02

    I know there has been some debate about my request to adding a feature to be able to filter within the body of the newsgroups.
    Its a option, if you do not like then do not use it. Filtering out the good and bad messages using only the subject line does
    not
    always work. Some words being filter within the body of the email for example porn, sex or horney and this type of material
    should be on your bbs system in the first place. Filtering by the subject should remain as a option, but adding a second
    filtering option is better. It givves the tools the sysop need to do their job and takes very small effort to do it.

    I was thinking it might be worthwhile to have a generic option to run an application or script against two files (headers and
    body) for messages
    being imported via qwk, ftn or newslink as well as smtp.

    That would work.

    ---
    ■ Synchronet ■ sbbs.dynu.net 2025
  • From Altere@VERT/ATHEL to plt on Tue Nov 10 20:01:52 2020
    Re: Newslink.js filtering
    By: plt to All on Tue Nov 10 2020 03:41 pm

    I know there has been some debate about my request to adding a feature to be able to filter within the body of the newsgroups. Its a option, if you do not like then do not use it. Filtering out the good and bad messages using only the subject line does not always work. Some words being filter within the body of the email for example porn, sex or horney and this type of material should be on your bbs system in the first place. Filtering by the subject should remain as a option, but adding a second filtering option is better. It givves the tools the sysop need to do their job and takes very small effort to do it.

    Well, then we wouldn't be able to have this conversation, it'd all be filtered. I don't see how adding another .can file with that implementation would be good for filtering the body of message. Let's throw a few examples out there in how I understand the way the .can files work, maybe I'm wrong but I just consulted the wiki again too.

    'sex' - this is going to filter out any msg with "sex" in it, maybe it's not spam?
    'sex~' - this is going to filter out any msg with "sexy", "sext", "sexes", etc. 'sex*' - this is going to filter out any msg like "unisex", "desex", etc..

    You need something far more complex then a .can file for this. But since this would only take 3hrs to code, according to you, that's been programming since 1986, why have you been talking about it for the last 48hrs instead of writting it?

    Anyhow, I'm no programmer, and never said it was a bad idea (although I have next to no use for this), I would request we be able to enable/disable spamc/spamassassin for varios message areas (sbbsecho maybe) to be able to set different reject thresholds per area and not just for the external mail processor. Then again, I would imagine that's probably not an easy task either. But as I said, I don't have much need for that as I only get an occasional spam msg through the mail processor, and don't have newgroups on my board so I don't see much spam in the message areas so I've never requested such feature. And not being a programmer, I'd have to sit and wait until someone implemented it for me. But you on the other hand, you should be able to tackle this.

    -altere

    ---
    ■ Synchronet ■ Athelstan BBS ■ athelstan.org ■ telnet:23 / ssh:2222
  • From plt@VERT/SBBS to Mortifis on Tue Nov 10 21:59:21 2020
    Re: Re: Newslink.js filtering
    By: Mortifis to Tracker1 on Tue Nov 10 2020 17:38:30

    On 11/10/2020 8:41 AM, plt wrote:

    I was thinking it might be worthwhile to have a generic option to run an application or script against two files (headers and
    body) for messages being imported via qwk, ftn or newslink as well as smtp.


    The only real issue I see with filtering the message body for key words is that 'innocent' messages (conversations) could get
    filtered out, thereby failing to deliver someone's reply ... so, even if this feature were implemented (which would have to be
    strict in its rules) I would not use it ... just sayin

    ~Mortifis

    It should not be a issue if it's done right.

    ---
    ■ Synchronet ■ sbbs.dynu.net 2025
  • From MRO@VERT/BBSESINF to plt on Tue Nov 10 22:46:06 2020
    Re: Re: Newslink.js filtering
    By: plt to Mortifis on Tue Nov 10 2020 09:59 pm


    I was thinking it might be worthwhile to have a generic option to
    run an application or script against two files (headers and body)
    for messages being imported via qwk, ftn or newslink as well as
    smtp.


    The only real issue I see with filtering the message body for key
    words is that 'innocent' messages (conversations) could get filtered
    out, thereby failing to deliver someone's reply ... so, even if this
    feature were implemented (which would have to be strict in its rules)
    I would not use it ... just sayin

    ~Mortifis

    It should not be a issue if it's done right.

    no, he fucking described exactly what the issues would be.

    just do not import old newsgroups full of spam.
    you're mentioning mi5 victim. i dont believe that insane person has posted for years.
    ---
    ■ Synchronet ■ ::: BBSES.info - free BBS services :::
  • From Mortifis@VERT/ALLEYCAT to MRO on Wed Nov 11 02:48:40 2020
    Re: Re: Newslink.js filtering
    By: plt to Mortifis on Tue Nov 10 2020 09:59 pm

    no, he fucking described exactly what the issues would be.

    Sorry, I was not able to see you're reply because my new plt_bot_omatic_fanity_filter.sem deleted it

    (_{!}_)

    :-P

    filter that, plt

    ---
    ­ Synchronet ­ AlleyCat! BBS Lake Echo, NS Canada
  • From plt@VERT/SBBS to Altere on Wed Nov 11 08:33:15 2020
    Re: Newslink.js filtering
    By: Altere to plt on Tue Nov 10 2020 20:01:52

    Re: Newslink.js filtering
    By: plt to All on Tue Nov 10 2020 03:41 pm


    Well, then we wouldn't be able to have this conversation, it'd all be filtered. I don't see how adding another .can file with th
    implementation would be good for filtering the body of message. Let's throw a few examples out there in how I understand the way
    the .can files work, maybe I'm wrong but I just consulted the wiki again too.

    Fitering within the body of the message is really a simple process since most computer lanuages already support string search function. The result of string result is happens very quick. For slowing down the process will depending on several factors, but pull the same newsgroup the 2nd time around if done correctly should not be a issue since newslink.js will start at the next message that was imported.


    'sex' - this is going to filter out any msg with "sex" in it, maybe it's not spam?
    'sex~' - this is going to filter out any msg with "sexy", "sext", "sexes", etc. 'sex*' - this is going to filter out any msg lik
    "unisex", "desex", etc..

    If you have users under the age of 18 then you should have this on your bbs in the first place.


    You need something far more complex then a .can file for this. But since this would only take 3hrs to code, according to you,
    that's been programming since 1986, why have you been talking about it for the last 48hrs instead of writting it?

    Anyhow, I'm no programmer, and never said it was a bad idea (although I have next to no use for this), I would request we be abl
    to enable/disable spamc/spamassassin for varios message areas (sbbsecho maybe) to be able to set different reject thresholds per
    area and not just for the external mail processor. Then again, I would imagine that's probably not an easy task either. But as I
    said, I don't have much need for that as I only get an occasional spam msg through the mail processor, and don't have newgroups
    my board so I don't see much spam in the message areas so I've never requested such feature. And not being a programmer, I'd hav
    to sit and wait until someone implemented it for me. But you on the other hand, you should be able to tackle this.

    IF people start doing their job by filtering and holding people accountable things get done.


    -altere

    ---
    ■ Synchronet ■ sbbs.dynu.net 2025
  • From plt@VERT/SBBS to Mortifis on Wed Nov 11 08:34:04 2020
    Re: Re: Newslink.js filtering
    By: plt to Mortifis on Tue Nov 10 2020 21:59:21

    Re: Re: Newslink.js filtering
    By: Mortifis to Tracker1 on Tue Nov 10 2020 17:38:30

    On 11/10/2020 8:41 AM, plt wrote:

    I was thinking it might be worthwhile to have a generic option to run an application or script against two files (headers
    and
    body) for messages being imported via qwk, ftn or newslink as well as smtp.


    The only real issue I see with filtering the message body for key words is that 'innocent' messages (conversations) could get
    filtered out, thereby failing to deliver someone's reply ... so, even if this feature were implemented (which would have to
    be
    strict in its rules) I would not use it ... just sayin

    Not true if it is done the correct way

    ---
    ■ Synchronet ■ sbbs.dynu.net 2025
  • From plt@VERT/SBBS to MRO on Wed Nov 11 08:42:43 2020
    Re: Re: Newslink.js filtering
    By: MRO to plt on Tue Nov 10 2020 22:46:06

    It should not be a issue if it's done right.

    no, he fucking described exactly what the issues would be.

    just do not import old newsgroups full of spam.
    you're mentioning mi5 victim. i dont believe that insane person has posted for years.

    Please go back and read the last messages. Its posted there.

    ---
    ■ Synchronet ■ sbbs.dynu.net 2025
  • From plt@VERT/SBBS to Mortifis on Wed Nov 11 08:43:42 2020
    Re: Re: Newslink.js filtering
    By: Mortifis to MRO on Wed Nov 11 2020 02:48:40

    Re: Re: Newslink.js filtering
    By: plt to Mortifis on Tue Nov 10 2020 09:59 pm

    no, he fucking described exactly what the issues would be.

    Sorry, I was not able to see you're reply because my new plt_bot_omatic_fanity_filter.sem deleted it

    (_{!}_)

    :-P

    filter that, plt

    Nice, the entire world is seeing how your acting like a little child now.

    ---
    ■ Synchronet ■ sbbs.dynu.net 2025
  • From Gamgee@VERT/PALANT to plt on Wed Nov 11 07:45:00 2020
    plt wrote to Mortifis <=-

    I was thinking it might be worthwhile to have a generic option to run an
    ap
    plication or script against two files (headers and
    body) for messages being imported via qwk, ftn or newslink as well as
    smtp

    The only real issue I see with filtering the message body for key words is
    th
    at 'innocent' messages (conversations) could get
    filtered out, thereby failing to deliver someone's reply ... so, even if
    this
    feature were implemented (which would have to be
    strict in its rules) I would not use it ... just sayin

    It should not be a issue if it's done right.

    Then do it.

    Right.

    After all, you've been a programmer since 1986, right?



    ... He does the work of 3 Men...Moe, Larry & Curly
    --- MultiMail/Linux v0.52
    ■ Synchronet ■ Palantir BBS * palantirbbs.ddns.net * Pensacola, FL
  • From Altere@VERT/ATHEL to plt on Wed Nov 11 08:47:55 2020
    Re: Newslink.js filtering
    By: plt to Altere on Wed Nov 11 2020 08:33 am

    Fitering within the body of the message is really a simple process since most computer lanuages already support string search function. The result of string result is happens very quick. For slowing down the process will depending on several factors, but pull the same newsgroup the 2nd time around if done correctly should not be a issue since newslink.js will start at the next message that was imported.

    You seem to have it all down, so where's your mods for everyone to test?

    'sex' - this is going to filter out any msg with "sex" in it, maybe
    it's not spam?
    'sex~' - this is going to filter out any msg with "sexy", "sext",
    "sexes", etc. 'sex*' - this is going to filter out any msg lik
    "unisex", "desex", etc..
    If you have users under the age of 18 then you should have this on your bbs in the first place.

    That completely flew right over your head, didn't it? First of all, I wouldn't put that in my subject.can (and certainly not a body.can if there was one) for examples I gave. I don't have underage people on my board, that I know of, but even if I did, what's the purpose of filtering SEXPOTS (that's what 'sex~' does) otherwise known as Synchronet External Plain Old Telephone System. The point is, that setup will filter a lot of messages that aren't spam. How can you NOT see that?

    mail processor, and don't have newgroups my board so I don't see much
    spam in the message areas so I've never requested such feature. And
    not being a programmer, I'd hav to sit and wait until someone
    implemented it for me. But you on the other hand, you should be able
    to tackle this.
    IF people start doing their job by filtering and holding people accountable things get done.

    I think you started this conversation up on Sunday in irc, so we're going 72hrs, plenty of time for you to write something up. So where is it? You've been programming since 1986 so you should have this knocked out already, yet you're still beating a dead horse. The idea got shot down by folks that actively develop and maintain sbbs, it's in your hands. Either do it, or just drop it already.

    -altere

    ---
    ■ Synchronet ■ Athelstan BBS ■ athelstan.org ■ telnet:23 / ssh:2222
  • From Altere@VERT/ATHEL to plt on Wed Nov 11 08:48:20 2020
    Re: Re: Newslink.js filtering
    By: plt to Mortifis on Wed Nov 11 2020 08:34 am

    Not true if it is done the correct way

    Then do it, the correct way.

    ---
    ■ Synchronet ■ Athelstan BBS ■ athelstan.org ■ telnet:23 / ssh:2222
  • From MRO@VERT/BBSESINF to Mortifis on Wed Nov 11 18:18:05 2020
    Re: Re: Newslink.js filtering
    By: Mortifis to MRO on Wed Nov 11 2020 02:48 am

    Re: Re: Newslink.js filtering
    By: plt to Mortifis on Tue Nov 10 2020 09:59 pm

    no, he fucking described exactly what the issues would be.

    Sorry, I was not able to see you're reply because my new plt_bot_omatic_fanity_filter.sem deleted it

    (_{!}_)

    :-P

    filter that, plt



    he said:

    " >> feature were implemented (which would have to be strict in its rules) >> I would not use it ... just sayin >> ~Mortifis pl> It should not be a issue if it's done right.
    "
    ---
    ■ Synchronet ■ ::: BBSES.info - free BBS services :::
  • From Tracker1@VERT/TRN to MRO on Wed Nov 11 17:36:56 2020
    On 11/10/2020 3:12 PM, MRO wrote:
    I was thinking it might be worthwhile to have a generic option to run an
    application or script against two files (headers and body) for messages
    being imported via qwk, ftn or newslink as well as smtp.

    Not sure if there's already something built in for this... and if there
    is, apologies. Just thinking something pretty generic could be used for
    this and beneficial. With the most generic action, of if either file is
    deleted, then it doesn't post, and puts a message out. This could be
    used to support virus scans, etc as well.

    i think years ago i asked rob how to filter newsgroup msgs for certain
    spam by editing the .js file. i forgot how to do it and dont have the
    old .js file.

    anyways, if you did the 2 different files and ran scripts on them it
    would really slow down msg import. depends on if you care about that.

    I know it's easy enough in newslink and nntp service js files, just
    thinking if it was a consistent implementation for all. The main point
    would be to allow for virus scanning to work correctly.

    As to slowing things down, maybe... modern SSDs are so much faster, as
    are computers than the old days, that on a modern system you likely
    wouldn't even notice the difference in practice.

    Beyond AV, being able to integrate better anti-spam with email and nntp
    would be beneficial as well. Training behaviors, etc. And while many
    would run without them, it would be a nice option.

    --
    Michael J. Ryan
    tracker1 +o Roughneck BBS
    ---
    ­ Synchronet ­ Roughneck BBS - roughneckbbs.com
  • From Tracker1@VERT/TRN to Altere on Wed Nov 11 17:42:16 2020
    On 11/10/2020 7:01 PM, Altere wrote:

    'sex' - this is going to filter out any msg with "sex" in it, maybe it's not spam?
    'sex~' - this is going to filter out any msg with "sexy", "sext", "sexes", etc.
    'sex*' - this is going to filter out any msg like "unisex", "desex", etc..

    This is part of an argument I used against adding a "language filter"
    for comments at a prior workplace...

    Positive: That's fucking bad-ass!
    Negative: May you swallow a huge pile of human excriment.

    Any language feature will catch messages that are positive and allow
    messages that are negative, which is the real desire.

    Of course signaling for spamminess is more complex and the simple
    filters will never give that.

    --
    Michael J. Ryan
    tracker1 +o Roughneck BBS
    ---
    ­ Synchronet ­ Roughneck BBS - roughneckbbs.com
  • From poindexter FORTRAN@VERT/REALITY to Tracker1 on Thu Nov 12 07:03:00 2020
    Tracker1 wrote to MRO <=-

    Beyond AV, being able to integrate better anti-spam with email and nntp would be beneficial as well. Training behaviors, etc. And while many would run without them, it would be a nice option.

    I miss Procmail, would love to be able to run Synchronet's netmail
    against it. I had a script called "Procmail Sanitizer" that would
    strip out HTML and make pretty readable plain-text email, as well as
    filtering out a lot of script/possibly hostile links,etc.



    ... Always the first steps
    --- MultiMail/XT v0.52
    ■ Synchronet ■ realitycheckBBS -- http://realitycheckBBS.org
  • From poindexter FORTRAN@VERT/REALITY to Tracker1 on Thu Nov 12 07:05:00 2020
    Tracker1 wrote to Altere <=-

    This is part of an argument I used against adding a "language filter"
    for comments at a prior workplace...

    Positive: That's fucking bad-ass!
    Negative: May you swallow a huge pile of human excriment.

    I remember one of the early discussions about Google's language
    parsing facilities. The speaker was talking about hot dogs, the
    devil being in the details. Are you talking about a concession at a
    baseball game or an animal locked in a car?



    ... Always the first steps
    --- MultiMail/XT v0.52
    ■ Synchronet ■ realitycheckBBS -- http://realitycheckBBS.org
  • From Tracker1@VERT/TRN to poindexter FORTRAN on Thu Nov 12 20:26:42 2020
    On 11/12/2020 8:03 AM, poindexter FORTRAN wrote:
    Beyond AV, being able to integrate better anti-spam with email and nntp
    would be beneficial as well. Training behaviors, etc. And while many
    would run without them, it would be a nice option.

    I miss Procmail, would love to be able to run Synchronet's netmail
    against it. I had a script called "Procmail Sanitizer" that would
    strip out HTML and make pretty readable plain-text email, as well as filtering out a lot of script/possibly hostile links,etc.

    Kind of what I was thinking... iirc you usually want a just the headers
    file, so that it can be done before processing the body, then one that includes the headers and body content so a full pass scan can run.

    It's been a few years since I worked on mail software at all though.

    --
    Michael J. Ryan
    tracker1 +o Roughneck BBS
    ---
    ­ Synchronet ­ Roughneck BBS - roughneckbbs.com
  • From Gene Buckle@VERT/RETROARC to plt on Thu Nov 12 08:57:47 2020
    Re: Re: Newslink.js filtering
    By: plt to Mortifis on Wed Nov 11 2020 08:34 am

    Not true if it is done the correct way

    So get on it.

    g.

    ---
    ■ Synchronet ■ The Retro Archive - bbs.retroarchive.org