Conversation:
Notices
-
@andstatus @verius The API I will be writing is entirely new. The existing endpoints won't change or be changed.
-
If you're a client-app developer or someone else that has a stake or interest in API use, the next few days are the time to tell me what you want to see in the new API for postActiv.
(Just to be clear, I'm keeping the old ones for compatability and just making an entirely new one to supplement it that others can use.)
-
@xj9 Well you can continue to use it then.
-
@archaeme It really isn't great for anything. It's designed to emulate a centralized system's API, and postActiv (and by extension GNU social) are not centralized, as I need not say.
-
@archaeme I could probably manage this on the backend, but Qvitter itself would have to be patched to follow it.
-
@archaeme I will probably implement the various endpoints Qvitter makes available that are extensions to the core API as part of this, where it makes sense anyways, but Qvitter itself's behaviour is outside the scope of postActiv.
-
@archaeme While I could patch Qvitter to do this, I kind of want to avoid getting sucked into patching a bunch of third-party products, for a bunch of reasons.
-
@archaeme This is also why I want to cut out the external libraries as much as is possible.
-
@lambadalambda I will copy this into my issue for the API so it's not forgotten.
One thing I do want to do with postActiv is make remote and local users as transparent in the backend as possible.
-
@lambadalambda I won't be changing the existing API.
-
@bob @lambadalambda Migration actually exists in the backend, but was made optional a long time ago due to security concerns. I am probably going to re-examine this however.
-
@bob Outputting the full YAML outline of an account is entirely possible, though it could be resource-consuming for a large account. That isn't an unsurmountable concern however and there is already a means to deal with large content in the codebase we can leverage towards this end.
The larger concern is devising a way to detect malicious edits or other suck injection vectors through the imported data.
-
@bob *such injections, though I would agree that injection attacks suck, yes
-
@lambadalambda I want to keep any changes to the existing API to plugins so that I don't touch the core behaviour and end up breaking compatability. I'd be willing to backport YAML versions to a Twitter API plugin though.
-
@maiyannah My main wish for GNU Social API improvement is to have URIs (globally unique identifiers) of users and messages everywhere, where now ”local" IDs are used. I see local ids usage as a major deficiency of current API. Which breaks federation (from a client point of view) and provides a small window - one "instance" of the network, via which the whole communication oc…
-
@andstatus @thefaico @archaeme Global IDs are definitely desirable but would be difficult to pull off without collissions unless you just did it namespaced by instance (which seems a bit like cheating and not really what you want). Improving transparency of remote notices and users is a good idea that I've been working at on and off but it's a complex problem since …
-
@xj9 @andstatus @thefaico @archaeme I don't have much interest in adding unneccesary external libraries to the software. As is the majority of GS's technical debt comes from having a bunch of very old external libraries, one of which hasn't been updated since 2009. Nonetheless, a hash could work.
-
@maiyannah I think that "namespaced by instance" is OK. E.g. user ID: http://status.hackerposse.com/url/6715 message ID: http://status.hackerposse.com/url/6715 The main point is that in order to request the same user or a message from different GNU Social instances, one should use the same Global ID. I.e. the same URL path will be used to request same message from any instance,…
-
@andstatus @thefaico I think I will have it available both by a hash and a namespaced notice_id
A hash is unique and therefore a better idea, but not really easily-supported in the older software (youd have to do something like have a modern remote generate one for a remote and then have it honoured) - but a namespaced notice_id would give us most of the benefits and still "work" in GS.
-
@andstatus @thefaico One important feature I'm going to build into the API is flexibility in what you request and how. I want it to be easy for clients to get JUST the data they need, as this will alleviate server load greatly (a single API request for a timeline right now can run over a hundred queries in some circumstances!) and also make things much easier for t…
-
@maiyannah 1. Conversations also need Globally unique IDs. They are local numbers also now. I'm currently testing usage of Conversation id to get full conversation in one request instead of "discovering" of previous posts one by one: and I'm really impressed by this feature of the API. 2. Another point of server load optimization: honoring since_id parameter in a search request…
-
@andstatus @thefaico Yes the API should definitely honour all search parameters. I'm not sure about fixing this in the old API (I will have to look but the moment you start pulling in one thread in old GS code, the whole tapestry starts to unravel...) but I will definitely make a point of ensuring this in the new code. For conversations and the other things we wan…
-
@andstatus @thefaico Do you have an email I could assign to a git account? If you are interested, I can set you up with an account on the postActiv gitlab so you can participate there if you like. It is currently on a kind of invite system because I got way too many spam signups.
-
It should be possible to make the conversation ID the same as the ID of the first message in that conversation. It means conversation IDs won't be sequential, but it'll make it easier (on the originating instance) to identify the start (conversationID == messageID)
-
@bobjonkman Mikael changed this, it was the original behaviour. There were a variety of problems related to this, not the least of which was the fact that the original post message may not be visible to your instance.
-
@maiyannah 1. I agree, adding new field into each "object" of the JSON message with UID in addition to each existing ID will be a good compromise preserving compatibility with old clients. 2. Regarding choice of the UID format/structure I strongly support a UID, which allows to identify easily the GNU Social instance that generated the UID. This way clients will be able to requ…
-
@maiyannah Thank you, I will definitely take part in discussions and in remote testing using AndStatus client. But currently I don't plan to setup my own server.
My personal email: yvolk1@gmail.com
-
@andstatus @thefaico Should have an email with a link to setup password etc soon!
-
@verius @andstatus @archaeme Yes this is why I think an ADDITIONAL uid field is a good idea but REPLACING the existing id would be a bad idea.
-
@pttr1 The idea is to make it so we have an id for a notice/user/etc that is unique across the entire network, not just on a local node.
-
@pttr1 It would make it easy for a compatible server that has the unique IDs to allow users to migrate between different servers, for one use case.
-
@pttr1 It makes a lot of instances where you're comparing data between servers a lot easier if you have an identifier you know is unique in the fediverse.
-
@pttr1 I'm not sure about mastodon, but in gnusocial it's a local notice_id. It will be unique on the local instance, but isn't gauranteed not to conflict with a notice_id on another one. In fact it almost assuredly will.
-
@pttr1 You got it. Thats what we're discussing fixing.
-
@pttr1 Well, lets use the example of a user migration script as our use case.
Right now, since all notices don't have unique ids, for every single notice a user has, you would have to re-process it, and assign each and every notice a new local id. For someone that has 10k notices, 100k notices, etc, that would be quite the processing task.
-
@pttr1 Indeed. And if we have unique IDs, we avoid that. There's also many other positives. For example, if we have ids for notices we know are unique, then if we have a conversation we're missing notices from we can request the specific notice ids we're missing. We can't really do this (easily) now because they're not unique. So if your server doesn't fede…
-
@pttr1 I have no idea how mastodon does it, but all notices in GNU social and postActiv are stored in a central notices table. The notice itself just contains a field that says who posted it, to reference the user.
-
@yvolk Anyone should be able to comment, but let me know if it doesn't work for you in some capacity.
The project is at https://git.postactiv.com/postActiv/postActiv/ - the API discussion specifically at https://git.postactiv.com/postActiv/postActiv/issues/63
-
@maiyannah 1. As we are thinking about good interoperability with !gnusocial instances that have old API only, I think we better add "local I'd part" not only to the newly formed "message/notice id", but to other IDs also. So a User UID will be: http://status.hackerposse.com/url/13370 i.e. it will include local user I'd as the third part of the UID 2. Regarding message/notice …
-
@maiyannah Regarding unique Message IDs and global messages' identification/matching/reduplication. It came to me that the root of messages duplication problem that we are periodically discussing, lies in the message ID generation at a server side. This is why messages, which were not posted yet from a client to a server, cannot be easily identified: they have to be matched bot…
-
@andstatus the URI (as in the context of Atom post id, which !GNUsocial uses as a transport format) cannot be assumed to have a discernible structure. It can be any character combination, or at least only as restricted as in the Atom standard. The URI is available in the post's raw atom formatted representation. It is probably still not available in the Twitter compati…
-
@mmn No problem with "opaque" URIs. They simply won't be parsed and hence will be useless for discovery of the !fediverse. Old clients/servers won't parse any URIs anyway.
Actually we are discussing future client API for GNU Social that @maiyannah is planning to implement. I'm referring to Twitter-compatible API only for comparison.
@verius
-
@andstatus @verius @mmn @thefaico He'd be surprised how many people are dead set on using the twitter API despite it's limitations, maybe :/
-
@verius @andstatus In that case it's happening because the client is sending a retry without checking if it doesn't already exist.
-
@verius @andstatus @thefaico I had more than one person comment telling me it was pointless to add something new because the twitter API was "good enough".
-
@verius @andstatus AndStatus will try resending a status if it doesn't think the server got it. But sometimes it did, and this results in duplication. Seems to mostly be a problem of untidy shutdowns or when its been suspended in the background.
-
@verius @andstatus Having notices have unique identifiers could probably help solve this.
-
@verius And that we're in the fediverse is why it falls apart and is insufficient.
-
@verius @andstatus Making it unique solves a variety of problems elsewhere as well. Id prefer to have it unique.
-
@verius @andstatus It also would be of use in allowing more seamless alt-account federation for the people that like having 98954 alts/
-
@verius @andstatus If the URI is a cryptographic hash of notice contents, it should be universally reproducable.
-
@verius @andstatus The more info that is hashed, the more unique the hash is going to be.
-
@verius @andstatus Let's use myself as an example. I have my main account here, and I also have alt accounts on other instances - wrongthink where I helped them restore their instance after a crash, and shitposter club. So say, I have those three accounts in AndStatus, because it makes it easy to combine the various timelines so I don't miss anything. I go to …
-
@verius @andstatus If we have a universal ID, and the client app is aware of it, it can look up the duplication challenge, and see it was posted by an account that is registered with the client, and go "oh yeah, this was something we tried to send earlier, looks like it did send"
-
@verius @andstatus (If they don't have the account registered, the client should probably discard the duplication challenge and respond saying, 'no we want to send this')
-
@verius @andstatus We're hashing to provide a unique identifier, not data integrity
(For proving integrity, I think signing with a checksum is probably better since an origin server can be considered authoratative for its own notices.)
-
@verius @andstatus I'm not really sure how to explain this in a more simple fashion than "we're using it make provide a unique identifier"
-
@verius @andstatus That might be a good idea but IIRC there's probably a better idea than using a wiki for that, I'll look into something tomorrow (later today?) after I've properly rested.
-
@maiyannah @verius @yvolk @pedro @bobjonkman @mmn @clacke @xj9 @pedro @takeshitakenji @archaeme @deavmi
maiyannah
AndStatus
Former Bob Jonkman -- Please use the new server at https://gs.jonkman.ca
MMN-o ✅⃠
All Hacker Poesy content and data are available under the