Blog

ISR part two: How our membership approach helps to preserve the integrity of the scholarly record

Amanda Bartell, Ginny Hendricks, Lucy Ofiesh – 2022 October 10

In Research IntegrityTrustworthinessMembershipOperationsGovernance

In part one of our series on the Integrity of the Scholarly Record (ISR), we talked about how the metadata that our members register with us helps to preserve the integrity of the record, and in particular how ’trust signals’ in the metadata, combined with relationships and context, can help the community assess the work. 

In this second blog, we describe membership eligibility and what you can and cannot tell simply from the fact that an organisation is a Crossref member; why increasing participation and reducing barriers actually helps to enhance the integrity of the scholarly record; and how we handle the very small number of cases where there may be a question mark.

Who can become a Crossref member and do we check new applicants?

Membership is open to organisations that “produce professional and scholarly materials and content”, and this is deliberately defined broadly. We’re a global community of members with content in all disciplines, in many formats, with all kinds of business models - research institutions, publishers, government agencies, research funders, banks, museums and many more. 

Essentially, if your content is likely to be cited in the research ecosystem and you consider it part of the evidence trail, then you’re eligible to join.

We ask organisations to complete an online application form and accept our member terms. On receipt of the application, we run a few very basic checks to ensure that:

1. The applicant can meet the membership criteria and seems to have the capacity to fulfill the obligations (and follow our code of conduct).
2. We are legally permitted to accept them as a member (for example, we can’t accept applications from some countries due to sanctions.
3. They haven’t previously been a member of Crossref whose membership was revoked.
4. They haven’t misrepresented themselves in the application (such as their location).
5. The applicant or an affiliate is not already a member of Crossref (so that we can advise they join under a single membership fee).

As long as the applicant can meet these requirements, and as long as they are able to pay any membership fees upfront for their first year of membership, they are able to become a Crossref member, get a DOI prefix, and start registering their metadata to share it with the global scholarly community. 

We are aware that some organisations in some regions may not be able to join Crossref independently. There may be barriers for them - the cost of membership fees, the fact that we only accept payment in US dollars, language barriers or technical barriers. 

To help increase participation globally, we work with sponsors in some regions. All sponsors facilitate membership for organisations who wish to participate in Crossref. They pay one central membership fee on behalf of all the members they work with, and they also pay content registration fees on behalf of their members. Many sponsors register content on behalf of their members, and even if they don’t, most provide local language and technical support. Sponsors are able to charge for their services, but it can be a very economical route for a member to join. In the last year, out of the 2,322 new members that we’ve welcomed, almost 58% joined via a sponsor.

We also waive registration fees for members in certain lower income countries who join via three of our sponsors, and we are planning to expand this program soon (pending board approval in November). [EDIT 2022-November-23: The new Global Equitable Membership (GEM) Program was approved and takes effect 1st January 2023]

The importance of keeping barriers to entry low 

As you can see, the checks that we run on new applicants are fairly limited in scope. In the last year, we’ve welcomed 2,322 new members and we only declined 39 applications. And 34 of these declined applications were effectively from one organisation whose membership was revoked in 2019.

Even this minimal set of checks takes a lot of research and keeps our member support specialists very busy - thank you Sally Jennings and Robbykha Rosalien (as well as contractors Kim and Collin). 

So why shouldn’t we run more extensive checks on new member applicants? Why don’t we check the quality of their content, or that they are following best practices? Why don’t we decline membership for organisations that can’t demonstrate editorial integrity or that aren’t meeting 100% of the membership obligations from the start?

Nevermind the additional capacity that more extensive checks on the over 200 applicants we receive per month would entail, it’s more fitting with our mission to:

1. enable equitable participation; and
2. focus on evidence:

Equitable participation

Inclusivity is very important to us - after all, one of our organisational truths (the guiding principles for everything we do) is “come one, come all”, and this is mirrored in the POSI principles that commit us to broad stakeholder representation. We know that for new organisations, it may take them a while to be able to completely fulfil the membership obligations. We support them with information to help them understand what being a participant in the Crossref community entails. These organisations would have less of a chance of developing better practices if we were to limit membership in Crossref to ‘proven’ candidates. Besides, it would introduce a race condition; if joining and sharing metadata through Crossref is widely considered best practice, new entrants need to join Crossref in order to show that they are adopting best practices.

Trust signals and the Research Nexus

Secondly, it’s not our role to make such a call; we don’t have the expertise to decide if an organisation would be considered “good” at what they are producing; there are other organisations guiding in this area, such as with the Principles of Transparency and Best Practice in Scholarly Publishing. Instead, we focus on the decision-making tools, metadata, and relationships that can help provide trust signals for the community.

Once members start registering their content, their activity and metadata speak about their practices – others in the community can process that metadata, combined with its wider context, and identify trust signals to make their own decisions. That metadata can only be shared in an open and machine-readable way if an organisation joins Crossref and starts registering their records and underpinning data with us. 

To paint a more detailed picture of the scholarly record, our priority is to get more and varied organisations contributing to the research nexus, rather than putting up barriers and blockers until they are performing perfectly. If they aren’t acting in the best interests of the scholarly community, then having the metadata available to assess will quickly make that obvious and hopefully encourage changes - sunlight being the best disinfectant, as the saying goes.

As we said in the first ISR blog:

“Crossref itself doesn’t assess the quality of content or the integrity of the research process but rather enables those who produce scholarly outputs to provide metadata (effectively evidence) about how they ensure the quality of content and how the outputs fit into the scholarly record.”

In our next post in the series, we’ll talk more about the workflow and decision-making tools we have in place and are planning to develop. We’ll pose questions about what kinds of metadata give what kind of levels of trustworthiness.

Helping new members become “good Crossref citizens”

Once an applicant becomes a member, we help them to completely fulfil the membership terms - ensuring that, for example, they register and display DOIs, keep their metadata up to date, and implement reference linking properly. 

We have a lot of documentation on our website, we run regular events and webinars, and we have a series of automated onboarding emails for new members to help them move through the key stages of the member journey from set up and onboarding to levelling up and using additional services like Crossmark and Similarity Check. Our staff are also on hand alongside Ambassadors and other members in our Community Forum. Speaking of POSI (and transparent operations) we receive around 3,000 emails per month with support requests so we are gradually moving support from closed 1:1 email to the more public and efficient community support forum.

We work with members who aren’t fulfilling the obligations to understand challenges and help explain what they need to do. This is currently reactive, but we have plans to automate checks on whether members are meeting the membership terms in future.

Outside of confirming that our members are behaving as “good Crossref citizens”, there aren’t many other areas where the membership team typically gets involved. Our mission is to help preserve the integrity of the scholarly record by making the metadata provided by our members openly available in a machine-readable format. We don’t investigate our members’ business practices or take a deep dive into their editorial processes (such as peer review), and there are many areas where we aren’t able to get involved. For example, we cannot arbitrate title ownership disputes.  

It’s all about preserving the integrity of the scholarly record

We do sometimes revoke membership, but this is for limited reasons: 

1. unpaid invoices;
2. legal sanctions or judgments against the member or its home country; or
3. contravention of the membership terms.

Membership revocation due to unpaid invoices

We spend a lot of time communicating with members who haven’t paid their invoices and ensuring they have the information they need to solve the problem. Revoking membership due to unpaid fees is an absolute last step for us, but financial sustainability means we can keep the organisation afloat and keep our infrastructure running.

Where members have unpaid fees, we eventually suspend their access to register new records and then ultimately revoke their membership if the fees remain unpaid. Once an organisation’s membership has been revoked, they would need to re-apply if they wanted to become a member again in the future. If accepted, the applicant would need to pay all outstanding invoices before re-joining. 

In March 2022, we revoked membership for around 140 members due to unpaid invoices (out of a total of over 17,000 active members).  

Membership revocation due to sanctions

Occasionally, we are informed of sanctions that we need to comply with, such as the recent case of Russia invading Ukraine where each Russian member needed to be checked for individual sanctions and some were revoked. Such revocations have to be voted on by the Executive Committee and then ratified by the board. Read more information on our sanctions process. 

Membership revocation for cause

Very occasionally there may be evidence that a member is in contravention of the membership terms. This may include:

• Misrepresentation in the original membership application
• Fraudulent use of identifiers or metadata
• Contravening the code of conduct
• Any other basis set forth in our governing documents.

We always try to work together with the member to solve problems, and again, revoking membership is an absolute last step. The revocation has to be voted on by the Executive Committee and then ratified by the board. 

Our first ever revocation for cause was in July 2019 for OMICS, after the board voted that the US Federal Trade Commission’s ruling against them amounted to a cause for revocation. There have been a handful of cases since. For example, most recently in September this year we revoked membership for a member who was registering DOIs for journals with the ISSNs of similarly-named publications.

There’s more information about our processes to revoke membership on our website.

More participation for the win

In conclusion, we believe that the more parties able to participate in Crossref and provide metadata and context for the research nexus, the more robust this makes the scholarly record.

But do you agree? Are these measures enough? What other information about our membership operations would help us be more transparent? As we said in our first blog, we need your help to establish whether our approach is still the right one, if we are missing anything and what else we might be able to do.

Here’s how you can help:

• Join the discussion about the integrity of the scholarly record on our community forum.
• Keep an eye out for future blog posts and meetings. We are having a small, in-person discussion prior to the Frankfurt Book Fair and will report on this in a future blog post.
• Sign up to attend Crossref LIVE22 for updates on these topics and all things Crossref.
• Join and support initiatives and organisations that we partner with or who use our metadata to look at ethical practices, for example, COPE, DOAJ, and OASPA, and review the Principles of Transparency in Scholarly Publishing, which these organisations worked on with WAME.